LOGIN
Sign In or Register
Avatar
Not Registered Yet?

Join Now! It's FREE. Get full access and benefit from this site

Reset My password - Remind Me My username

Username
Password
Remember me
This is the time of DevOps in software industry and DevOps uses different different languages for deployment automation and for software development. This is the reason if you are a DevOps professional and want to be succeed in DevOps role than command on scripting languages is must. But, one can not be a master of all. Right? Therefore, In this article I am going to share a list of top 10 scripting languages which will be useful for your DevOps journey.
 
But before that let's have a quick look on Scripting language.
 
Scripting languages are programming languages that communicate and integrate with other programming languages. In other words, scripting languages controlls interactive programs operations by giving it sequence of work to execute by one command at a time.
 
Now, let's look on to the list of top 10 scripting languages
 
1. Microsoft PowerShell


 
Microsoft powershell or powershell is belongs to Microsoft and an open source cross platform scripting language. If you have application infrastructure with windows than powershell is must for you. This scripting language is designed for system admins. 
 
 
2. Puppet



Puppet is a configuration management tool and it has it's own declarative language to describe system configuration. It runs on Linux, Unix-like and also on Windows. This one is available under Apache 2.7.0 and General Pubic license.  Puppet uses a custom declarative language, it just needs to specify ‘what’ action needs to be performed on the resources.
 
3. Chef



Chef is basically known as configuration management which belongs to chef is also a scripting language which is designed by David Morgan. It makes programs look like cooking any food.
 
4. Bash



Bash is basically a command language which is available in open source and written by Brian Fox in the year 1989. It can read scripts and Bash is the most commonly used Unix shell. Bash supports Linux, Windows and Mac OS.
 
5. Ruby



Ruby is amongst one of the best programming language but it is also a scripting language which is written by Yukihiro Matsumoto in the year 1995. Ruby supports cross platforms and it is available under GPL and BSD license. It supports multiple programming paradigms, including functional, object-oriented, and imperative.
 
 
6. Ansible



Ansible is known as configuration management and application deployment tool but it is also amongst top scripting languages. This language is belongs to Ansible Inc. and written by their community members. It supports Linux, Unix-like and Windows operating system.
 
7. Perl



Perl is a scripting language which is used for advanced web applications development. Perl is written by Larry Wall and first released in the year 1987. Perl supports cross platforms. Perl is available under general public license. 
 
8. Python


Python is also amongst the top scripting languages which is used for high level of programming. It was first released in the year 1991 by Guido van Rossum. python is available under Python Software Foundation License. It's supports Cross-platform.
 
9. Go language



Go scripting language belongs to the Internet giant Google. This scripting language is written by Robert Griesemer, Rob Pike and Ken Thompson and it was released in the year 2009. It supports  Linux, macOS, FreeBSD, NetBSD, OpenBSD, Windows, Plan 9, DragonFly BSD and Solaris operating systems. It is available in open source.
 
10. Groovy



Groovy can be used as a scripting language and it is also consider as a top scripting languages for DevOps professionals. It was designed by James Strachan and developed by Guillaume Laforge, Jochen Theodorou, Paul King and Cedric Champeau. It was first released in the year 2003 and available under Apache license. It supports Java platform.
 
 
Do you agree with this list? If not than feel free to respond in the comment box with your own take on the most essential scripting languages. One more thing, I would like to add here, if you need help to learn all these scripting languages and DevOps courses than scmGalaxy can help you in this. scmGalaxy is a community of DevOps professionals who are well experienced in this domain. 

Commonly Used Sign- Punctuation and their Name in Perl


( => ) Comma Arrow

 ( ‘ ’, “ ”, ' ', " " ) à Quotation marks

( $ ) àDollar sign

( ~ ) à Tilde

( % ) àPercent sign

,  ) à Comma

( @ ) à At Sign

( ; ) à Semicolon

( & ) à Ampersand Sign

( : ) à Colon

( / ) à Forward Slash/Slash sign

( , , ) à Dash

( \ )  à Backslash/Slash Sign

( - )    à Hyphen-minus

( < )  à Less than Sign

( _ ) àunderscore

( > )  à Greater than sign

( / ) à slash/stroke/solidus

( { } )  àCurly Braces

( * ) à Asterisk

( [  ] ) à Square brackets

( ^ ) à caret       

( ( ) )  à Round brackets

( ¦, | ) à vertical bar/broken bar/pipe

( , , ) àprime

( ) à Ditto mark

( ! ) à Exclamation Mark

( ? ) - Question Mark

 

 

Any Server will (mostly) do   

  • CGI will not work without a server
  • Your Server must allow CGI
  • Your CGI application must be written in a language supported by your server
  • You can use probably your existing web server
  • If you need to test on your local computer, you may install WAMP/XAMPP
  • Each Server has a particular requirements about running  CGI Applications


Prerequisties
Experience in relevant programming language

  • Perl
  • Php
  • Unix Shell
  • c/c++
  • HTML/CSS


What is CGI
Common Interface Gateway

  • How web browsers submit forms and interact with programs on the server
  • Used for simple interaction applications
  • Can be used with any programming language
  • Often called “CGI Scripts”
  • Can be written in any language
  • Run in the server, not in a web browser
  • Must follow server requirements for running applocations


ANY SERVER WILL (MOSTLY) do

  • CGI will not work without a web server
  • Your Server must allow CGI
  • Your CGI application must be written in a language supported by your server
  • You can use  your existing web server
  • If you need to test on your local computer, you may install WAMP/XAMPP


Some Guidelines for Unix/Apapche users

  1. Permission – 755 ( aka rwxr-xr-x)
  2. Filename associates
  3. .cgi, .pl, .php …etc
  4. .httaccess
  5. Options+ExecCGI
  6. AddHandle cgi-scripts .cgi
  7. SetHander cgi-script
  8. Access to server log


Some Guidelines for Windows Servers
Many Windows servers do not support perl or PHP
May require ASP or C#
Usually require specific filename extension
.pl for perl
.php for php
.asp or .aspx for ASP
Access to error log
Check with your server web site or support staff

How CGI works

  1. Browser Communicates with server over HTTP
  2. Browser sends a request to the server consisting of header and optional body
  3. Server sends information to the CGI application
  4. Environment variables
  5. Body may be sent in the STDIN stream
  6. Server sends a response consisting of a header and a body



There are many ways to execute external commands from Perl. The most commons are:

  • system function
  • exec function
  • backticks (``) operator
  • open function

All of these methods have different behaviour, so you should choose which one to use depending of your particular need. In brief, these are the recommendations:


method

use if ...

system()

you want to execute a command and don't want to capture its output

exec

you don't want to return to the calling perl script

backticks

you want to capture the output of the command

open

you want to pipe the command (as input or output) to your script


More detailed explanations of each method follows:

Using system()
system() executes the command specified. It doesn't capture the output of the command.
system() accepts as argument either a scalar or an array. If the argument is a scalar, system() uses a shell to execute the command ("/bin/sh -c command"); if the argument is an array it executes the command directly, considering the first element of the array as the command name and the remaining array elements as arguments to the command to be executed.
For that reason, it's highly recommended for efficiency and safety reasons (specially if you're running a cgi script) that you use an array to pass arguments to system()

Example:  
   #-- calling 'command' with  arguments
   system("command arg1 arg2 arg3");
    
   #-- better way of calling the same command
   system("command", "arg1", "arg2",  "arg3"); 
   The return value  is set in $?; this value is the exit status of the command as returned  by the 'wait' call; to get the real exit status of the command you have to  shift right by 8 the value of $? ($? >> 8). 
   If the value of $? is -1, then the command failed to execute, in that case you may check the value  of $! for the reason of the failure.
 
Example:  
   system("command",  "arg1");
   if ( $? == -1 )
   {
     print "command failed: $!\n";
   }
   else
   {
     printf "command exited with value %d", $? >> 8;
   } 
 

Using exec()
The exec() function executes the command specified and never returns to the calling program, except in the case of failure because the specified command does not exist AND the exec argument is an array.
Like in system(), is recommended to pass the arguments of the functions as an array.

Using backticks (``)
In this case the command to be executed is surrounded by backticks. The command is executed and the output of the command is returned to the calling script.
In scalar context it returns a single (possibly multiline) string, in list context it returns a list of lines or an empty list if the command failed.
The exit status of the executed command is stored in $? (see system() above for details).

Example:  
   #-- scalar context
   $result = `command arg1 arg2`;
    
   #-- the same command in list context
   @result = `command arg2 arg2`; 
   Notice that the  only output captured is STDOUT, to collect messages sent to STDERR you should  redirect STDERR to STDOUT
 
Example:  
   #-- capture STDERR as well  as STDOUT
   $result = `command 2>&1`; 
 

Using open()
Use open() when you want to:
- capture the data of a command (syntax: open("command |"))
- feed an external command with data generated from the Perl script (syntax: open("| command"))

Examples:  
   #-- list the processes  running on your system
   open(PS,"ps -e -o pid,stime,args |") || die "Failed: $!\n";
   while ( <PS> )
   {
     #-- do something here
   }
    
   #-- send an email to user@localhost
   open(MAIL, "| /bin/mailx -s test user\@localhost ") || die  "mailx failed: $!\n";
 print MAIL "This is a test message"; 

 

|| use Vs require in perl || What is the difference between use and require?

Except of course that use is evaluated at compile time where as require is evaluated at run time in other word, A use anywhere in the code will be evaluated when the code is run compiled, but require - import's can only get evaluated when encoutered.

The differences are many and often subtle:

  1. use only expects a bareword, require can take a bareword or an expression
  2. use is evaluated at compile-time, require at run-time
  3. use implicitly calls the import method of the module being loaded, require does not
  4. use excepts arguments in addition to the bareword (to be passed to import), require does not
  5. use does not behave like a function (i.e can't be called with parens, can't be used in an expression, etc), whereas require does

do $file is like eval `cat $file`, except the former:
1.1: searches @INC and updates %INC.
1.2: bequeaths an *unrelated* lexical scope on the eval'ed code.

require $file is like do $file, except the former:
2.1: checks for redundant loading, skipping already loaded files.
2.2: raises an exception on failure to find, compile, or execute $file.

require Module is like require "Module.pm", except the former:
3.1: translates each "::" into your system's directory separator.
3.2: primes the parser to disambiguate class Module as an indirect object.

use Module is like require Module, except the former:
4.1: loads the module at compile time, not run-time.
4.2: imports symbols and semantics from that package to the current one.

Command to learn more about use and require
> perldoc -f require
> perldoc -f use

What is epoch time?

The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT), not counting leap seconds (in ISO 8601: 1970-01-01T00:00:00Z). Literally speaking the epoch is Unix time 0 (midnight 1/1/1970), but 'epoch' is often used as a synonym for 'Unix time'. Many Unix systems store epoch dates as a signed 32-bit integer, which might cause problems on January 19, 2038 (known as the Year 2038 problem or Y2038).

 

Human readable time Seconds
1 hour 3600 seconds
1 day 86400 seconds
1 week 604800 seconds
1 month (30.44 days) 2629743 seconds
1 year (365.24 days) 31556926 seconds

How to get the current epoch time in ...

Perl time
PHP time()
Ruby Time.now (or Time.new). To display the epoch: Time.now.to_i
Python import time first, then time.time()
Java long epoch = System.currentTimeMillis()/1000;
Microsoft .NET C# epoch = (DateTime.Now.ToUniversalTime().Ticks - 621355968000000000) / 10000000;
VBScript/ASP DateDiff("s", "01/01/1970 00:00:00", Now())
R as.numeric(Sys.time())
Erlang calendar:datetime_to_gregorian_seconds(calendar:now_to_universal_time( now()))-719528*24*3600.
MySQL SELECT unix_timestamp(now()) More info (+ negative epochs)
PostgreSQL SELECT extract(epoch FROM now());
Oracle PL/SQL SELECT (SYSDATE - TO_DATE('01/01/1970 00:00:00', 'MM-DD-YYYY HH24:MI:SS')) *
24 * 60 * 60 FROM DUAL
SQL Server SELECT DATEDIFF(s, '1970-01-01 00:00:00', GETUTCDATE())
JavaScript Math.round(new Date().getTime()/1000.0) getTime() returns time in milliseconds.
Tcl/Tk clock seconds
Unix/Linux Shell date +%s
PowerShell Get-Date -UFormat "%s" Produces: 1279152364.63599
Other OS's Command line: perl -e "print time" (If Perl is installed on your system)

Convert from human readable date to epoch

Perl Use the Perl Epoch routines
PHP mktime(hour, minute, second, month, day, year) More info
Ruby Time.local(year, month, day, hour, minute, second, usec ) (or Time.gm for GMT/UTC input). To display add .to_i
Python import time first, then int(time.mktime(time.strptime('2000-01-01 12:34:00', '%Y-%m-%d %H:%M:%S'))) - time.timezone
Java long epoch = new java.text.SimpleDateFormat ("MM/dd/yyyy HH:mm:ss").parse("01/01/1970 01:00:00").getTime();
VBScript/ASP DateDiff("s", "01/01/1970 00:00:00", time field) More info
C Use the C Epoch Converter routines
R as.numeric(as.POSIXct("MM/dd/yyyy HH:mm:ss", origin="1970-01-01"))
MySQL SELECT unix_timestamp(time) Time format: YYYY-MM-DD HH:MM:SS or YYMMDD or YYYYMMDD
More on using Epoch timestamps with MySQL
PostgreSQL SELECT extract(epoch FROM date('2000-01-01 12:34'));
With timestamp: SELECT EXTRACT(EPOCH FROM TIMESTAMP WITH TIME ZONE '2001-02-16 20:38:40-08');
With interval: SELECT EXTRACT(EPOCH FROM INTERVAL '5 days 3 hours');
SQL Server SELECT DATEDIFF(s, '1970-01-01 00:00:00', time field)
JavaScript Use the JavaScript Date object
Unix/Linux Shell date +%s -d"Jan 1, 1980 00:00:01" Replace '-d' with '-ud' to input in GMT/UTC time.

Convert from epoch to human readable date

Perl Use the Perl Epoch routines
PHP date(output format, epoch); Output format example: 'r' = RFC 2822 date More info
Ruby Time.at(epoch)
Python import time first, then time.strftime("%a, %d %b %Y %H:%M:%S +0000", time.localtime(epoch)) Replace time.localtime with time.gmtime for GMT time. More info
Java String date = new java.text.SimpleDateFormat("MM/dd/yyyy HH:mm:ss").format(new java.util.Date (epoch*1000));
VBScript/ASP DateAdd("s", epoch, "01/01/1970 00:00:00") More info
C Use the C Epoch Converter routines
R as.POSIXct(epoch, origin="1970-01-01")
MySQL from_unixtime(epoch, optional output format) The default output format is YYY-MM-DD HH:MM:SS more ...
PostgreSQL PostgreSQL version 8.1 and higher: SELECT to_timestamp(epoch); More info Older versions: SELECT TIMESTAMP WITH TIME ZONE 'epoch' + epoch * INTERVAL '1 second';
Oracle PL/SQL SELECT to_date('01-JAN-1970','dd-mon-yyyy')+(1326357743/60/60/24) from dual
Replace 1326357743 with epoch.
SQL Server DATEADD(s, epoch, '1970-01-01 00:00:00')
Microsoft Excel =(A1 / 86400) + 25569 Format the result cell for date/time, the result will be in GMT time (A1 is the cell with the epoch number). For other time zones: =((A1 +/- time zone adjustment) / 86400) + 25569.
Crystal Reports DateAdd("s", {EpochTimeStampField}-14400, #1/1/1970 00:00:00#) -14400 used for Eastern Standard Time. See Time Zones.
JavaScript Use the JavaScript Date object
Tcl/Tk clock format 1325376000 More info
Unix/Linux Shell date -d @1190000000 Replace 1190000000 with your epoch, needs recent version of 'date'. Replace '-d' with '-ud' for GMT/UTC time.
PowerShell Function get-epochDate ($epochDate) { [timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').AddSeconds($epochDate)) }, then use: get-epochDate 1279152364. Works for Windows PowerShell v1 and v2
Other OS's Command line: perl -e "print scalar(localtime(epoch))" (If Perl is installed) Replace 'localtime' with 'gmtime' for GMT/UTC time.

Reference

http://www.epochconverter.com/

 

Checking File Attributes in Perl

I have been using perl for quite some time now. I have also been using the file handling logic in my scripts. However, what I did not know till now is that you can quickly check for certain file properties in perl. Here is an example:
#!/usr/bin/perl
my (@description,$size);
if (-e $file)
{
push @description, ‘binary’ if (-B _);
push @description, ‘a socket’ if (-S _);
push @description, ‘a text file’ if (-T _);
push @description, ‘a block special file’ if (-b _);
push @description, ‘a character special file’ if (-c _);
push @description, ‘a directory’ if (-d _);
push @description, ‘executable’ if (-x _);
push @description, (($size = -s _)) ? “$size bytes” : ‘empty’;
print “$file is “, join(‘, ‘,@description),”\n”;
}
Nice, isn’t it?
Here is the complete list of features that you can check:


Operator

Description

-A

Age of file (at script startup) in days since modification.

-B

Is it a binary file?

-C

Age of file (at script startup) in days since modification.

-M

Age of file (at script startup) in days since modification.

-O

Is the file owned by the real user ID?

-R

Is the file readable by the real user ID or real group?

-S

Is the file a socket?

-T

Is it a text file?

-W

Is the file writable by the real user ID or real group?

-X

Is the file executable by the real user ID or real group?

-b

Is it a block special file?

-c

Is it a character special file?

-d

Is the file a directory?

-e

Does the file exist?

-f

Is it a plain file?

-g

Does the file have the setgid bit set?

-k

Does the file have the sticky bit set?

-l

Is the file a symbolic link?

-o

Is the file owned by the effective user ID?

-p

Is the file a named pipe?

-r

Is the file readable by the effective user or group ID?

-s

Returns the size of the file, zero size = empty file.

-t

Is the filehandle opened by a TTY (terminal)?

-u

Does the file have the setuid bit set?

-w

Is the file writable by the effective user or group ID?

-x

Is the file executable by the effective user or group ID?

-z

Is the file size zero?

 

To find a program not on this list (and there are hundreds), try the man program with the -k option followed by a keyword, for example,
man -k tape
to find all on-line manual entries that refer to programs that work with tapes.

 

Terminal control characters for C-shell (csh)

^h, backspace erase previously typed character
^u erase entire line of input so far typed
^d end-of-input for programs reading from terminal
^s suspend writing to terminal (freezes terminal)
^q continue writing to terminal
^z suspend currently running job; restart with bg or fg
^c kill currently running program and allow clean-up before exiting
^\ emergency kill of currently running program with no chance of cleanup

Also see a list of special characters that should not be used in filenames.

Login and authentication

login access computer; start interactive session
logout disconnect terminal session
passwd change local login password; you must set a strong password that is not easily guessed
kinit obtain kerberos ticket for connections to other kerberized computers
kdestroy destroy kerberos tickets (authorizations)

Information

date show date and time
history list of previously executed commands
man show online documentation by program name
info online documentation for GNU programs
w, who who is on the system and what they are doing
whoami who is logged onto this terminal
top show system stats and top CPU using processes
uptime show one line summary of system status

File management

cat combine files
cp copy files
ls list files in a directory and their attributes
mv change file name or directory location
rm remove files
ln create another link (name) to a file
chmod set file permissions
crypt encode/decode a file with a private key
gzip, gunzip compress/decompress a file
find find files that match specific criteria

Display contents of files

cat copy files to display device
more show text file on display terminal with paging control
head show first few lines of a file(s)
tail show last few lines of a file; or reverse line order
vi full-featured screen editor for modifying text files
pico simple screen editor for modifying text files
grep display lines that match a pattern
lpr send file to printer
pr format file with page headers, multiple columns, etc.
diff compare two files and show differences
cmp compare two binary files and report if different
comm compare two files; show common or unique lines
od display binary files as eqivalent octal/hex codes
strings show printable text embedded in binary files
file examine file(s) and guess type: text, data, program, etc.
wc count characters, words, and lines in a file

Directories

cd change to new directory
mkdir create new directory
rmdir remove empty directory (you must remove files first)
mv change name of directory
pwd show current directory

Disks

df summarize free space on disk filesystems
du show disk space used by files or directories

Special character handling for C-shell (See man csh)

* match any characters in a file name
~user shorthand for home directory of user
$name substitute value of variable name
\ turn off special meaning of character that follows
' in pairs, quote string with special chars, except !
" in pairs, quote string with special chars, except !, $
` in pairs, substitute output from enclosed command

Controlling program execution for C-shell (See man csh)

& run job in background
^c kill job in foreground
^z suspend job in foreground
fg restart suspended job in foreground
bg run suspended job in background
; delimit commands on same line
() group commands on same line
! re-run earlier commands from history list
jobs list current jobs
ps show process information
kill kill background job or previous process
nice run program at lower priority
at run program at a later time
crontab run program at specified intervals
limit see or set resource limits for programs
alias create alias name for program (normally used in .login file)
sh, csh execute command file

Controlling program input/output for C-shell (See man csh)

| pipe output to input
> redirect output to a storage file
< redirect input from a storage file
>> append redirected output to a storage file
tee copy input to both file and next program in pipe
script make file record of all terminal activity

Email and communication

msgs read system bulletin board messages
mailx send/read email; can be run by other programs to send exisiting files via email
uuencode
uudecode
encode/decode a binary file for transmission via email
rn read USENET news groups

Editors and formatting utilities

sed programmable text editor for data streams
vi full-featured editor for character terminals
emacs GNU emacs editor for character terminals
xemacs GNU emacs editor for X Window terminals
pico very simple text editor
fmt fill and break lines to make all same length
fold break long lines to specified length

X Window client programs (output to X terminal or server)

xterm provide login shell window
xauth manipulate authorization files
xload show system load
xman full screen online manual viewer
xemacs GNU emacs editor
gv interface to contol gs to display PostScript or PDF files on screen
xdvi display DVI files on X Window (screen preview)
gnuplot interactive data plotting on screen

TeX typesetting system

tex process TeX files to DVI (device independent) output
latex process LaTeX files to DVI
dvips print DVI files on Postscript laser printer
xdvi display DVI files on X Window (screen preview)
latex2html translate LaTeX files to HTML (for web pages)

Printing (BSD based)

lpr send file to print queue
lpq examine status of files in print queue
lprm remove a file from print queue
enscript convert text files to PostScript format for printing

Interpreted languages and data manipulation utilities

sed programmable text editor for data streams
awk pattern scanning and processing language
perl Practical Extraction and Report Language
sort sort or merge lines in a file(s) by specified fields
tr translate characters
cut cut out columns from a file
paste paste columns into a file
dd copy data between devices; reblock; convert EBCDIC

Graphics and mapping

gnuplot interactive data plotting; outputs to PostScript or X Window
gs "ghostscript" converter displays PostScript files on X Window displays or other devices

Networking/communications

ssh remote login/command execution; encrypted
scp remote non-interactive file copy; encrypted
sftp remote interactive file copy; encrypted
telnet remote network login; plain text password - not recommended
ftp network file transfer program; plain text password - not recommended
host find IP address for given host name, or vice versa
lynx web browser for character based (text-only) terminals
gzip, gunzip compress/decompress a file
tar combine multiple files/dirs into single archive
uuencode
uudecode
encode/decode a binary file for transmission via email

Compilers, interpreters and programming tools

csh command language interpreter (C-shell scripts)
ksh command language interpreter (Korn-shell scripts)
sh command language interpreter (Borne-shell scripts)
f77 Fortran 77 compiler
f2c convert fortran source code to C source code
gcc GNU C compiler
g++ GNU C++ compiler
dbx command-line symbolic debugger for compiled C or Fortran
make recompile programs from modified source
cflow generate C flow graph

Programming libraries (see man library_name)

lapack Fortran 77 routines for numerical linear algebra (supersedes LINPACK and EISPACK)
X routines to interface with X window system (no man page -- get the X Toolkit book)
dbm database routines
xdr library routines for external data representation
netcdf routines for machine independent data representation

Tape manipulation and archiving

mt manipulate tape drive and position tape
dd unformatted tape read and write; file conversion
tar archive disk files on tape or disk
ltf read/write ANSI standard label tapes
A parameter is an entity that stores values. It can be a name, a number or some special characters.

Bash shell provides two kind of parameters.

Positional Parameter and Special Parameter

Bash Positional Parameter – $0, $1, $2 ..

Positional parameters are the arguments given to your scripts when it is invoked. It could be from $1 to $N. When N consists of more than a single digit, it must be enclosed in a braces like ${N}.

The variable $0 is the basename of the program as it was called.

The following example gets two arguments and provides arithmetic operations result between those two integers.

First, create the arithmetic.sh shell script as shown below.

$ cat arithmetic.sh

#!/bin/bash

echo -e "\$1=$1"

echo -e "\$2=$2"


let add=$1+$2

let sub=$1-$2

let mul=$1*$2

let div=$1/$2

echo -e "Addition=$add\nSubtraction=$sub\nMultiplication=$mul\nDivision=$div\n"

Next, execute the arithmetic.sh with proper parameters as shown below.

$ ./arithmetic.sh 12 10

$1=12

$2=10

Addition=22

Subtraction=2

Multiplication=120

Division=1

In the above output $1 has the value 12, and $2 has 10.

Shell built-in ‘let’ allows arithmetic operation to be performed on shell variables. The above script does the arithmetic operations such as addition, subtraction, multiplication and division on the given parameters.

Set / Unset Bash Positional Parameters

The built in set command is used to set and unset the positional parameter.

First, create the positional.sh shell script as shown below.

$ cat positional.sh

#!/bin/bash

# From command line

echo -e "Basename=$0"

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# From Set builtin

set First Second Third

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# Store positional parameters with -(hyphen)

set - -f -s -t

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# Unset positional parameter

set --

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

The above script prints the command line arguments first, then set command sets the positional parameter explicitly. Set with the – refers end of options, all following arguments are positional parameter even they can begin with ‘-’. Set with ‘–’ without any other arguments unset all the positional parameters.

Next, execute the positional.sh as shown below.

$ ./positional.sh

Basename=positional.sh

$1=12

$2=10

$3=

$1=First

$2=Second

$3=Third

$1=-f

$2=-s

$3=-t

$1=

$2=

$3=

Use Bash $* and $@ to Expand Positional Parameters

This example shows the value available in $* and $@.

First, create the expan.sh as shown below.

$ cat expan.sh

#!/bin/bash

export IFS='-'

cnt=1

# Printing the data available in $*

echo "Values of \"\$*\":"

for arg in "$*"

do

echo "Arg #$cnt= $arg"

let "cnt+=1"

done

cnt=1

# Printing the data available in $@

echo "Values of \"\$@\":"

for arg in "$@"

do

echo "Arg #$cnt= $arg"

let "cnt+=1"

done

Next, execute the expan.sh as shown below to see how $* and $@ works.

$ ./expan.sh "This is" 2 3

Values of "$*":

Arg #1= This is-2-3

Values of "$@":

Arg #1= This is

Arg #2= 2

Arg #3= 3

· The above script exported the value of IFS (Internal Field Separator) with the ‘-’.

· There are three parameter passed to the script expan.sh $1=”This is”,$2=”2″ and $3=”3″.

· When printing the each value of special parameter “$*”, it gives only one value which is the whole positional parameter delimited by IFS.

· Whereas “$@” gives you each parameter as a separate word.

Use $# to Count Positional Parameters

$# is the special parameter in bash which gives you the number of positional parameter in decimal.

First, create the arithmetic.sh as shown below.

$ cat arithmetic.sh

#!/bin/bash

if [ $# -lt 2 ]

then

echo "Usage: $0 arg1 arg2"

exit

fi

echo -e "\$1=$1"

echo -e "\$2=$2"

let add=$1+$2

let sub=$1-$2

let mul=$1*$2

let div=$1/$2

echo -e "Addition=$add\nSubtraction=$sub\nMultiplication=$mul\nDivision=$div\n"

If the number of positional parameters is less than 2, it will throw the usage information as shown below,

$ ./arithemetic.sh 10

Usage: ./arithemetic.sh arg1 arg2

Process related Parameters – $$ and $!

The special parameter $$ will give the process ID of the shell. $! gives you the process id of the most recently executed background process.

The following script prints the process id of the shell and last execute background process ID.

$ cat proc.sh

#!/bin/bash

echo -e "Process ID=$$"

sleep 1000 &

echo -e "Background Process ID=$!"

Now, execute the above script, and check the process id which its printing.

$ ./proc.sh

Process ID=9502

Background Process ID=9503

$ ps

PID TTY TIME CMD

5970 pts/1 00:00:00 bash

9503 pts/1 00:00:00 sleep

9504 pts/1 00:00:00 ps

$

Other Bash Special Parameters – $?, $-, $_

· $? Gives the exit status of the most recently executed command.

· $- Options set using set builtin command

· $_ Gives the last argument to the previous command. At the shell startup, it gives the absolute filename of the shell script being executed.

$ cat others.sh

#!/bin/bash

echo -e "$_"; ## Absolute name of the file which is being executed

/usr/local/bin/dbhome # execute the command.

#check the exit status of dbhome

if [ "$?" -ne "0" ]; then

echo "Sorry, Command execution failed !"

fi

echo -e "$-"; #Set options - hB

echo -e $_ # Last argument of the previous command.

In the above script, the last echo statement “echo -e $_” ($ underscore) also prints hB which is the value of last argument of the previous command. So $_ will give the value after expansion

$ ./others.sh

./others.sh

/home/oracle

Sorry, Command execution failed !

hB

hB

UNIX SED Introduction

· Sed is a “non-interactive” stream-oriented editor. Since its an “non-interactive” it can be used to automate editing.

· This allows you to edit multiple files, or to perform common editing operations without opening VI or emacs.

· Sed reads from a file or from its standard input and outputs to its standard output.

· Sed has two buffers which are called pattern buffer and hold buffer. Both are initially empty.

UNIX Sed Working Method

This is called as one execution cycle. Cycle continues till end of file/input is reached.

1. Read a entire line from stdin/file.

2. Removes any trailing newline.

3. Places the line, in its pattern buffer.

4. Modify the pattern buffer according to the supplied commands

5. Print the pattern buffer to stdout.

Printing Operation in Sed

Linux Sed command allows you to print only specific lines based on the line number or pattern matches. “p” is a command for printing the data from the pattern buffer.

Syntax:

# sed -n 'ADDRESS'p filename

# sed -n '/PATTERN/p' filename

Create the txt file sample.txt that will be used in all examples below.

# cat sample.txt

1. Linux - Sysadmin, Scripting etc.

2. Databases - Oracle, mySQL etc.

3. Hardware

4. Security (Firewall, Network, Online Security etc)

5. Storage

6. Cool gadgets and websites

7. Productivity (Too many technologies to explore)

8. Website Design

9. Software Development

10.Windows- Sysadmin, reboot etc.

Sed Address Format 1: NUMBER

This will only match Nth line in the input.

#sed –n ‘N’p filename

For example, 3p prints 3rd line of the input file sample.txt as below

# sed -n '3'p thegeekstuff.txt

3. Hardware

Sed Address Format 2: NUMBER1~NUMBER2

M~N with “p” command prints every Nth line starting from line M.

# sed -n ‘M~N’p filename

For example, 3~2p prints every 2nd line starting from 3rd line as shown below.

# sed -n '3~2'p thegeekstuff.txt
3. Hardware
5. Storage
7. Productivity (Too many technologies to explore)
9. Software Development

Sed Address Format 3: START,END

M,N with “p” command prints Mth line to Nth line.

# sed -n ‘M,N’p filename

For example, 4,8p prints from 4th line to 8th line from input file thegeekstuff.txt

# sed -n '4,8'p thegeekstuff.txt

4. Security (Firewall, Network, Online Security etc)

5. Storage

6. Cool gadgets and websites

7. Productivity (Too many technologies to explore)

8. Website Design

Sed Address Format 4: ‘$’ Last Line

$ with “p” command matches only the last line from the input.

# sed -n ‘$’p filename

For example, $p prints only the last line as shown below.

# sed -n '$'p thegeekstuff.txt
10.Windows- Sysadmin, reboot etc.
 

Sed Address Format 5: NUMBER,$

N,$ with “p” command prints from Nth line to end of file.

# sed -n ‘N,$p’ filename

For example 4,$p prints from 4th line to end of file.

# sed -n '4,$p' thegeekstuff.txt
4. Security (Firewall, Network, Online Security etc)
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)
8. Website Design
9. Software Development
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 1: PATTERN

PATTERN could be unix regular expression. The below command prints only the line which matches the given pattern.

# sed -n /PATTERN/p filename

For example, following prints the line only which matches the pattern “Sysadmin”.

# sed -n /Sysadmin/p thegeekstuff.txt
1. Linux - Sysadmin, Scripting etc.
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 2: /PATTERN/,ADDRESS

# sed -n ‘/PATTERN/,Np’ filename

For example, following prints lines which matches the pattern to Nth line, from input. 3rd line matches the pattern “Hardware”, so it prints from 3rd line to 6th line.

# sed -n '/Hardware/,6p' thegeekstuff.txt
3. Hardware
4. Security (Firewall, Network, Online Security etc)
5. Storage
6. Cool gadgets and websites

Sed Pattern Format 3: ADDRESS,/PATTERN/

It prints from the Nth line of the input, to the line which matches the pattern. If the pattern doesnt match, it prints upto end of the input.

# sed -n ‘N,/PATTERN/p’ filename

For example, 4th line matches the pattern “Security”, so it prints from 3rd line to 4th line.

# sed -n '3,/Security/p' thegeekstuff.txt
3. Hardware
4. Security (Firewall, Network, Online Security etc)

Sed Pattern Format 4: /PATTERN/,$

It prints from the line matches the given pattern to end of file.

# sed -n ‘/PATTERN/,$p’ filename

# sed -n '/Website/,$p' thegeekstuff.txt
8. Website Design
9. Software Development
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 5: /PATTERN/,+N

It prints the lines which matches the pattern and next N lines following the matched line.

# sed -n ‘/PATTERN/,+Np’ filename

For example, following prints the 5th line which matches the pattern /Storage/ and next two lines following /Storage/.

# sed -n '/Storage/,+2p' thegeekstuff.txt
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)

Sed Pattern Format 6: /PATTERN/,/PATTERN/

Prints the section of file between two regular expression (including the matched line ).

# sed -n ‘/P1/,/P2/p’ filename

For example, 5th line matches “Storage” and 8th line matches “Design”, so it prints 5th to 8th.

# sed -n '/Storage/,/Design/p' thegeekstuff.txt
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)
8. Website Design

Delete File Lines Using Address and pattern in Sed

“p” command prints the buffer (Remember to use “-n” option with “p”)

“d” command is just opposite, it’s for deletion. “d” will delete the pattern space buffer and immediately starts the next cycle.

Syntax:

# sed 'ADDRESS'd filename

# sed /PATTERN/d filename

Delete the Nth line.

“Nd” delete the Nth line and prints the other lines.

For example, 3d deletes 3rd line and prints other lines.

$sed 3d sample.txt

Delete Starting from 3rd line and every 2nd line from there.

$sed ‘3~2d’ sample.txt

Delete from 4ht to 8th line from file

$sed ‘4,8d’ sample.txt

Delete the last line from input

$sed ‘$d’ sample.txt

Delete the line which matches the given pattern from input

$sed /Sysadmin/d sample.txt

Delete the line from which matches the given pattern to end of the file

$sed ‘/Website/ ,$d’ sample.txt

Delete the line from which matches the given pattern and 2 lines next to that

$sed ‘Storage/ ,+2d’ sample.txt

Delete blank line from a file using sed.

$sed ‘/^$/d’ sample.txt

Page 1 of 2