Friday, 17 May 2013 09:37

Introduction of cgi

Written by

Any Server will (mostly) do   

  • CGI will not work without a server
  • Your Server must allow CGI
  • Your CGI application must be written in a language supported by your server
  • You can use probably your existing web server
  • If you need to test on your local computer, you may install WAMP/XAMPP
  • Each Server has a particular requirements about running  CGI Applications


Prerequisties
Experience in relevant programming language

  • Perl
  • Php
  • Unix Shell
  • c/c++
  • HTML/CSS


What is CGI
Common Interface Gateway

  • How web browsers submit forms and interact with programs on the server
  • Used for simple interaction applications
  • Can be used with any programming language
  • Often called “CGI Scripts”
  • Can be written in any language
  • Run in the server, not in a web browser
  • Must follow server requirements for running applocations


ANY SERVER WILL (MOSTLY) do

  • CGI will not work without a web server
  • Your Server must allow CGI
  • Your CGI application must be written in a language supported by your server
  • You can use  your existing web server
  • If you need to test on your local computer, you may install WAMP/XAMPP


Some Guidelines for Unix/Apapche users

  1. Permission – 755 ( aka rwxr-xr-x)
  2. Filename associates
  3. .cgi, .pl, .php …etc
  4. .httaccess
  5. Options+ExecCGI
  6. AddHandle cgi-scripts .cgi
  7. SetHander cgi-script
  8. Access to server log


Some Guidelines for Windows Servers
Many Windows servers do not support perl or PHP
May require ASP or C#
Usually require specific filename extension
.pl for perl
.php for php
.asp or .aspx for ASP
Access to error log
Check with your server web site or support staff

How CGI works

  1. Browser Communicates with server over HTTP
  2. Browser sends a request to the server consisting of header and optional body
  3. Server sends information to the CGI application
  4. Environment variables
  5. Body may be sent in the STDIN stream
  6. Server sends a response consisting of a header and a body



Main Responsibilities of the Linux System Administrator is as follows:

  1. Maintaining User Accounts
  2. Security
  3. Managing the disk space and Backup

The Administrator’s Privileges:

  1. Change the contents or attributes of any file like its permission and ownership
  2. He can delete any file eith rm even if it is write-protected.
  3. Initiate or Kill Any Process
  4. Change any user’s password without knowing the existing one
  5. Set the system clock with date
  6. Limit the maximum size of files that users are permitted to create with ulimlit
  7. Controls users access to the scheduling services like at and cron
  8. Control users access to many networking services like Telnet, FTP, etc

Commands:

su: Acquiring Superuser Status

# su  :- Acquire  Superuser after entering password but home directory does not change
   # su – l :– Acquire Superuser access after password but home  directory change to ROOT
 # su  - henry : - To  Login and Recreate a user environement

date – Setting the system Date

# date MMDDhhmm

wall: Communicating with users

# wall
   The machine will be shutdown today at 14 30
   The Back up will be at 12:30
 [Ctrl-d]

ulimit: Setting Limits on File Size

ulimit 6576565 Measured in 512 byte blocks

passwd – Changing any Password

passwd
 # passwd henry

Notes:

  1. Most administrative commands are resident in /sbin and /usr/sbin
  2. Linux uses Bash for normal and system administrative operations. Using Korn or C shell, all command may not work.
  3. # in prompt indicates that the login is from root user.
Friday, 04 May 2012 11:02

Executing external commands using perl

Written by

There are many ways to execute external commands from Perl. The most commons are:

  • system function
  • exec function
  • backticks (``) operator
  • open function

All of these methods have different behaviour, so you should choose which one to use depending of your particular need. In brief, these are the recommendations:


method

use if ...

system()

you want to execute a command and don't want to capture its output

exec

you don't want to return to the calling perl script

backticks

you want to capture the output of the command

open

you want to pipe the command (as input or output) to your script


More detailed explanations of each method follows:

Using system()
system() executes the command specified. It doesn't capture the output of the command.
system() accepts as argument either a scalar or an array. If the argument is a scalar, system() uses a shell to execute the command ("/bin/sh -c command"); if the argument is an array it executes the command directly, considering the first element of the array as the command name and the remaining array elements as arguments to the command to be executed.
For that reason, it's highly recommended for efficiency and safety reasons (specially if you're running a cgi script) that you use an array to pass arguments to system()

Example:  
   #-- calling 'command' with  arguments
   system("command arg1 arg2 arg3");
    
   #-- better way of calling the same command
   system("command", "arg1", "arg2",  "arg3"); 
   The return value  is set in $?; this value is the exit status of the command as returned  by the 'wait' call; to get the real exit status of the command you have to  shift right by 8 the value of $? ($? >> 8). 
   If the value of $? is -1, then the command failed to execute, in that case you may check the value  of $! for the reason of the failure.
 
Example:  
   system("command",  "arg1");
   if ( $? == -1 )
   {
     print "command failed: $!\n";
   }
   else
   {
     printf "command exited with value %d", $? >> 8;
   } 
 

Using exec()
The exec() function executes the command specified and never returns to the calling program, except in the case of failure because the specified command does not exist AND the exec argument is an array.
Like in system(), is recommended to pass the arguments of the functions as an array.

Using backticks (``)
In this case the command to be executed is surrounded by backticks. The command is executed and the output of the command is returned to the calling script.
In scalar context it returns a single (possibly multiline) string, in list context it returns a list of lines or an empty list if the command failed.
The exit status of the executed command is stored in $? (see system() above for details).

Example:  
   #-- scalar context
   $result = `command arg1 arg2`;
    
   #-- the same command in list context
   @result = `command arg2 arg2`; 
   Notice that the  only output captured is STDOUT, to collect messages sent to STDERR you should  redirect STDERR to STDOUT
 
Example:  
   #-- capture STDERR as well  as STDOUT
   $result = `command 2>&1`; 
 

Using open()
Use open() when you want to:
- capture the data of a command (syntax: open("command |"))
- feed an external command with data generated from the Perl script (syntax: open("| command"))

Examples:  
   #-- list the processes  running on your system
   open(PS,"ps -e -o pid,stime,args |") || die "Failed: $!\n";
   while ( <PS> )
   {
     #-- do something here
   }
    
   #-- send an email to user@localhost
   open(MAIL, "| /bin/mailx -s test user\@localhost ") || die  "mailx failed: $!\n";
 print MAIL "This is a test message"; 

 

Thursday, 05 April 2012 11:41

Difference between use and require in perl

Written by

|| use Vs require in perl || What is the difference between use and require?

Except of course that use is evaluated at compile time where as require is evaluated at run time in other word, A use anywhere in the code will be evaluated when the code is run compiled, but require - import's can only get evaluated when encoutered.

The differences are many and often subtle:

  1. use only expects a bareword, require can take a bareword or an expression
  2. use is evaluated at compile-time, require at run-time
  3. use implicitly calls the import method of the module being loaded, require does not
  4. use excepts arguments in addition to the bareword (to be passed to import), require does not
  5. use does not behave like a function (i.e can't be called with parens, can't be used in an expression, etc), whereas require does

do $file is like eval `cat $file`, except the former:
1.1: searches @INC and updates %INC.
1.2: bequeaths an *unrelated* lexical scope on the eval'ed code.

require $file is like do $file, except the former:
2.1: checks for redundant loading, skipping already loaded files.
2.2: raises an exception on failure to find, compile, or execute $file.

require Module is like require "Module.pm", except the former:
3.1: translates each "::" into your system's directory separator.
3.2: primes the parser to disambiguate class Module as an indirect object.

use Module is like require Module, except the former:
4.1: loads the module at compile time, not run-time.
4.2: imports symbols and semantics from that package to the current one.

Command to learn more about use and require
> perldoc -f require
> perldoc -f use

Tuesday, 22 November 2011 14:14

Checking File Attributes in Perl

Written by

Checking File Attributes in Perl

I have been using perl for quite some time now. I have also been using the file handling logic in my scripts. However, what I did not know till now is that you can quickly check for certain file properties in perl. Here is an example:
#!/usr/bin/perl
my (@description,$size);
if (-e $file)
{
push @description, ‘binary’ if (-B _);
push @description, ‘a socket’ if (-S _);
push @description, ‘a text file’ if (-T _);
push @description, ‘a block special file’ if (-b _);
push @description, ‘a character special file’ if (-c _);
push @description, ‘a directory’ if (-d _);
push @description, ‘executable’ if (-x _);
push @description, (($size = -s _)) ? “$size bytes” : ‘empty’;
print “$file is “, join(‘, ‘,@description),”\n”;
}
Nice, isn’t it?
Here is the complete list of features that you can check:


Operator

Description

-A

Age of file (at script startup) in days since modification.

-B

Is it a binary file?

-C

Age of file (at script startup) in days since modification.

-M

Age of file (at script startup) in days since modification.

-O

Is the file owned by the real user ID?

-R

Is the file readable by the real user ID or real group?

-S

Is the file a socket?

-T

Is it a text file?

-W

Is the file writable by the real user ID or real group?

-X

Is the file executable by the real user ID or real group?

-b

Is it a block special file?

-c

Is it a character special file?

-d

Is the file a directory?

-e

Does the file exist?

-f

Is it a plain file?

-g

Does the file have the setgid bit set?

-k

Does the file have the sticky bit set?

-l

Is the file a symbolic link?

-o

Is the file owned by the effective user ID?

-p

Is the file a named pipe?

-r

Is the file readable by the effective user or group ID?

-s

Returns the size of the file, zero size = empty file.

-t

Is the filehandle opened by a TTY (terminal)?

-u

Does the file have the setuid bit set?

-w

Is the file writable by the effective user or group ID?

-x

Is the file executable by the effective user or group ID?

-z

Is the file size zero?

 

Wednesday, 05 January 2011 07:04

Common unix commands and utilities

Written by

To find a program not on this list (and there are hundreds), try the man program with the -k option followed by a keyword, for example,
man -k tape
to find all on-line manual entries that refer to programs that work with tapes.

 

Terminal control characters for C-shell (csh)

^h, backspace erase previously typed character
^u erase entire line of input so far typed
^d end-of-input for programs reading from terminal
^s suspend writing to terminal (freezes terminal)
^q continue writing to terminal
^z suspend currently running job; restart with bg or fg
^c kill currently running program and allow clean-up before exiting
^\ emergency kill of currently running program with no chance of cleanup

Also see a list of special characters that should not be used in filenames.

Login and authentication

login access computer; start interactive session
logout disconnect terminal session
passwd change local login password; you must set a strong password that is not easily guessed
kinit obtain kerberos ticket for connections to other kerberized computers
kdestroy destroy kerberos tickets (authorizations)

Information

date show date and time
history list of previously executed commands
man show online documentation by program name
info online documentation for GNU programs
w, who who is on the system and what they are doing
whoami who is logged onto this terminal
top show system stats and top CPU using processes
uptime show one line summary of system status

File management

cat combine files
cp copy files
ls list files in a directory and their attributes
mv change file name or directory location
rm remove files
ln create another link (name) to a file
chmod set file permissions
crypt encode/decode a file with a private key
gzip, gunzip compress/decompress a file
find find files that match specific criteria

Display contents of files

cat copy files to display device
more show text file on display terminal with paging control
head show first few lines of a file(s)
tail show last few lines of a file; or reverse line order
vi full-featured screen editor for modifying text files
pico simple screen editor for modifying text files
grep display lines that match a pattern
lpr send file to printer
pr format file with page headers, multiple columns, etc.
diff compare two files and show differences
cmp compare two binary files and report if different
comm compare two files; show common or unique lines
od display binary files as eqivalent octal/hex codes
strings show printable text embedded in binary files
file examine file(s) and guess type: text, data, program, etc.
wc count characters, words, and lines in a file

Directories

cd change to new directory
mkdir create new directory
rmdir remove empty directory (you must remove files first)
mv change name of directory
pwd show current directory

Disks

df summarize free space on disk filesystems
du show disk space used by files or directories

Special character handling for C-shell (See man csh)

* match any characters in a file name
~user shorthand for home directory of user
$name substitute value of variable name
\ turn off special meaning of character that follows
' in pairs, quote string with special chars, except !
" in pairs, quote string with special chars, except !, $
` in pairs, substitute output from enclosed command

Controlling program execution for C-shell (See man csh)

& run job in background
^c kill job in foreground
^z suspend job in foreground
fg restart suspended job in foreground
bg run suspended job in background
; delimit commands on same line
() group commands on same line
! re-run earlier commands from history list
jobs list current jobs
ps show process information
kill kill background job or previous process
nice run program at lower priority
at run program at a later time
crontab run program at specified intervals
limit see or set resource limits for programs
alias create alias name for program (normally used in .login file)
sh, csh execute command file

Controlling program input/output for C-shell (See man csh)

| pipe output to input
> redirect output to a storage file
< redirect input from a storage file
>> append redirected output to a storage file
tee copy input to both file and next program in pipe
script make file record of all terminal activity

Email and communication

msgs read system bulletin board messages
mailx send/read email; can be run by other programs to send exisiting files via email
uuencode
uudecode
encode/decode a binary file for transmission via email
rn read USENET news groups

Editors and formatting utilities

sed programmable text editor for data streams
vi full-featured editor for character terminals
emacs GNU emacs editor for character terminals
xemacs GNU emacs editor for X Window terminals
pico very simple text editor
fmt fill and break lines to make all same length
fold break long lines to specified length

X Window client programs (output to X terminal or server)

xterm provide login shell window
xauth manipulate authorization files
xload show system load
xman full screen online manual viewer
xemacs GNU emacs editor
gv interface to contol gs to display PostScript or PDF files on screen
xdvi display DVI files on X Window (screen preview)
gnuplot interactive data plotting on screen

TeX typesetting system

tex process TeX files to DVI (device independent) output
latex process LaTeX files to DVI
dvips print DVI files on Postscript laser printer
xdvi display DVI files on X Window (screen preview)
latex2html translate LaTeX files to HTML (for web pages)

Printing (BSD based)

lpr send file to print queue
lpq examine status of files in print queue
lprm remove a file from print queue
enscript convert text files to PostScript format for printing

Interpreted languages and data manipulation utilities

sed programmable text editor for data streams
awk pattern scanning and processing language
perl Practical Extraction and Report Language
sort sort or merge lines in a file(s) by specified fields
tr translate characters
cut cut out columns from a file
paste paste columns into a file
dd copy data between devices; reblock; convert EBCDIC

Graphics and mapping

gnuplot interactive data plotting; outputs to PostScript or X Window
gs "ghostscript" converter displays PostScript files on X Window displays or other devices

Networking/communications

ssh remote login/command execution; encrypted
scp remote non-interactive file copy; encrypted
sftp remote interactive file copy; encrypted
telnet remote network login; plain text password - not recommended
ftp network file transfer program; plain text password - not recommended
host find IP address for given host name, or vice versa
lynx web browser for character based (text-only) terminals
gzip, gunzip compress/decompress a file
tar combine multiple files/dirs into single archive
uuencode
uudecode
encode/decode a binary file for transmission via email

Compilers, interpreters and programming tools

csh command language interpreter (C-shell scripts)
ksh command language interpreter (Korn-shell scripts)
sh command language interpreter (Borne-shell scripts)
f77 Fortran 77 compiler
f2c convert fortran source code to C source code
gcc GNU C compiler
g++ GNU C++ compiler
dbx command-line symbolic debugger for compiled C or Fortran
make recompile programs from modified source
cflow generate C flow graph

Programming libraries (see man library_name)

lapack Fortran 77 routines for numerical linear algebra (supersedes LINPACK and EISPACK)
X routines to interface with X window system (no man page -- get the X Toolkit book)
dbm database routines
xdr library routines for external data representation
netcdf routines for machine independent data representation

Tape manipulation and archiving

mt manipulate tape drive and position tape
dd unformatted tape read and write; file conversion
tar archive disk files on tape or disk
ltf read/write ANSI standard label tapes
Friday, 10 December 2010 15:18

Shell Script Parameters

Written by
A parameter is an entity that stores values. It can be a name, a number or some special characters.

Bash shell provides two kind of parameters.

Positional Parameter and Special Parameter

Bash Positional Parameter – $0, $1, $2 ..

Positional parameters are the arguments given to your scripts when it is invoked. It could be from $1 to $N. When N consists of more than a single digit, it must be enclosed in a braces like ${N}.

The variable $0 is the basename of the program as it was called.

The following example gets two arguments and provides arithmetic operations result between those two integers.

First, create the arithmetic.sh shell script as shown below.

$ cat arithmetic.sh

#!/bin/bash

echo -e "\$1=$1"

echo -e "\$2=$2"


let add=$1+$2

let sub=$1-$2

let mul=$1*$2

let div=$1/$2

echo -e "Addition=$add\nSubtraction=$sub\nMultiplication=$mul\nDivision=$div\n"

Next, execute the arithmetic.sh with proper parameters as shown below.

$ ./arithmetic.sh 12 10

$1=12

$2=10

Addition=22

Subtraction=2

Multiplication=120

Division=1

In the above output $1 has the value 12, and $2 has 10.

Shell built-in ‘let’ allows arithmetic operation to be performed on shell variables. The above script does the arithmetic operations such as addition, subtraction, multiplication and division on the given parameters.

Set / Unset Bash Positional Parameters

The built in set command is used to set and unset the positional parameter.

First, create the positional.sh shell script as shown below.

$ cat positional.sh

#!/bin/bash

# From command line

echo -e "Basename=$0"

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# From Set builtin

set First Second Third

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# Store positional parameters with -(hyphen)

set - -f -s -t

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

# Unset positional parameter

set --

echo -e "\$1=$1"

echo -e "\$2=$2"

echo -e "\$3=$3"

The above script prints the command line arguments first, then set command sets the positional parameter explicitly. Set with the – refers end of options, all following arguments are positional parameter even they can begin with ‘-’. Set with ‘–’ without any other arguments unset all the positional parameters.

Next, execute the positional.sh as shown below.

$ ./positional.sh

Basename=positional.sh

$1=12

$2=10

$3=

$1=First

$2=Second

$3=Third

$1=-f

$2=-s

$3=-t

$1=

$2=

$3=

Use Bash $* and $@ to Expand Positional Parameters

This example shows the value available in $* and $@.

First, create the expan.sh as shown below.

$ cat expan.sh

#!/bin/bash

export IFS='-'

cnt=1

# Printing the data available in $*

echo "Values of \"\$*\":"

for arg in "$*"

do

echo "Arg #$cnt= $arg"

let "cnt+=1"

done

cnt=1

# Printing the data available in $@

echo "Values of \"\$@\":"

for arg in "$@"

do

echo "Arg #$cnt= $arg"

let "cnt+=1"

done

Next, execute the expan.sh as shown below to see how $* and $@ works.

$ ./expan.sh "This is" 2 3

Values of "$*":

Arg #1= This is-2-3

Values of "$@":

Arg #1= This is

Arg #2= 2

Arg #3= 3

· The above script exported the value of IFS (Internal Field Separator) with the ‘-’.

· There are three parameter passed to the script expan.sh $1=”This is”,$2=”2″ and $3=”3″.

· When printing the each value of special parameter “$*”, it gives only one value which is the whole positional parameter delimited by IFS.

· Whereas “$@” gives you each parameter as a separate word.

Use $# to Count Positional Parameters

$# is the special parameter in bash which gives you the number of positional parameter in decimal.

First, create the arithmetic.sh as shown below.

$ cat arithmetic.sh

#!/bin/bash

if [ $# -lt 2 ]

then

echo "Usage: $0 arg1 arg2"

exit

fi

echo -e "\$1=$1"

echo -e "\$2=$2"

let add=$1+$2

let sub=$1-$2

let mul=$1*$2

let div=$1/$2

echo -e "Addition=$add\nSubtraction=$sub\nMultiplication=$mul\nDivision=$div\n"

If the number of positional parameters is less than 2, it will throw the usage information as shown below,

$ ./arithemetic.sh 10

Usage: ./arithemetic.sh arg1 arg2

Process related Parameters – $$ and $!

The special parameter $$ will give the process ID of the shell. $! gives you the process id of the most recently executed background process.

The following script prints the process id of the shell and last execute background process ID.

$ cat proc.sh

#!/bin/bash

echo -e "Process ID=$$"

sleep 1000 &

echo -e "Background Process ID=$!"

Now, execute the above script, and check the process id which its printing.

$ ./proc.sh

Process ID=9502

Background Process ID=9503

$ ps

PID TTY TIME CMD

5970 pts/1 00:00:00 bash

9503 pts/1 00:00:00 sleep

9504 pts/1 00:00:00 ps

$

Other Bash Special Parameters – $?, $-, $_

· $? Gives the exit status of the most recently executed command.

· $- Options set using set builtin command

· $_ Gives the last argument to the previous command. At the shell startup, it gives the absolute filename of the shell script being executed.

$ cat others.sh

#!/bin/bash

echo -e "$_"; ## Absolute name of the file which is being executed

/usr/local/bin/dbhome # execute the command.

#check the exit status of dbhome

if [ "$?" -ne "0" ]; then

echo "Sorry, Command execution failed !"

fi

echo -e "$-"; #Set options - hB

echo -e $_ # Last argument of the previous command.

In the above script, the last echo statement “echo -e $_” ($ underscore) also prints hB which is the value of last argument of the previous command. So $_ will give the value after expansion

$ ./others.sh

./others.sh

/home/oracle

Sorry, Command execution failed !

hB

hB

Tuesday, 23 November 2010 09:49

Unix Sed Quick Reference

Written by

UNIX SED Introduction

· Sed is a “non-interactive” stream-oriented editor. Since its an “non-interactive” it can be used to automate editing.

· This allows you to edit multiple files, or to perform common editing operations without opening VI or emacs.

· Sed reads from a file or from its standard input and outputs to its standard output.

· Sed has two buffers which are called pattern buffer and hold buffer. Both are initially empty.

UNIX Sed Working Method

This is called as one execution cycle. Cycle continues till end of file/input is reached.

1. Read a entire line from stdin/file.

2. Removes any trailing newline.

3. Places the line, in its pattern buffer.

4. Modify the pattern buffer according to the supplied commands

5. Print the pattern buffer to stdout.

Printing Operation in Sed

Linux Sed command allows you to print only specific lines based on the line number or pattern matches. “p” is a command for printing the data from the pattern buffer.

Syntax:

# sed -n 'ADDRESS'p filename

# sed -n '/PATTERN/p' filename

Create the txt file sample.txt that will be used in all examples below.

# cat sample.txt

1. Linux - Sysadmin, Scripting etc.

2. Databases - Oracle, mySQL etc.

3. Hardware

4. Security (Firewall, Network, Online Security etc)

5. Storage

6. Cool gadgets and websites

7. Productivity (Too many technologies to explore)

8. Website Design

9. Software Development

10.Windows- Sysadmin, reboot etc.

Sed Address Format 1: NUMBER

This will only match Nth line in the input.

#sed –n ‘N’p filename

For example, 3p prints 3rd line of the input file sample.txt as below

# sed -n '3'p thegeekstuff.txt

3. Hardware

Sed Address Format 2: NUMBER1~NUMBER2

M~N with “p” command prints every Nth line starting from line M.

# sed -n ‘M~N’p filename

For example, 3~2p prints every 2nd line starting from 3rd line as shown below.

# sed -n '3~2'p thegeekstuff.txt
3. Hardware
5. Storage
7. Productivity (Too many technologies to explore)
9. Software Development

Sed Address Format 3: START,END

M,N with “p” command prints Mth line to Nth line.

# sed -n ‘M,N’p filename

For example, 4,8p prints from 4th line to 8th line from input file thegeekstuff.txt

# sed -n '4,8'p thegeekstuff.txt

4. Security (Firewall, Network, Online Security etc)

5. Storage

6. Cool gadgets and websites

7. Productivity (Too many technologies to explore)

8. Website Design

Sed Address Format 4: ‘$’ Last Line

$ with “p” command matches only the last line from the input.

# sed -n ‘$’p filename

For example, $p prints only the last line as shown below.

# sed -n '$'p thegeekstuff.txt
10.Windows- Sysadmin, reboot etc.
 

Sed Address Format 5: NUMBER,$

N,$ with “p” command prints from Nth line to end of file.

# sed -n ‘N,$p’ filename

For example 4,$p prints from 4th line to end of file.

# sed -n '4,$p' thegeekstuff.txt
4. Security (Firewall, Network, Online Security etc)
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)
8. Website Design
9. Software Development
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 1: PATTERN

PATTERN could be unix regular expression. The below command prints only the line which matches the given pattern.

# sed -n /PATTERN/p filename

For example, following prints the line only which matches the pattern “Sysadmin”.

# sed -n /Sysadmin/p thegeekstuff.txt
1. Linux - Sysadmin, Scripting etc.
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 2: /PATTERN/,ADDRESS

# sed -n ‘/PATTERN/,Np’ filename

For example, following prints lines which matches the pattern to Nth line, from input. 3rd line matches the pattern “Hardware”, so it prints from 3rd line to 6th line.

# sed -n '/Hardware/,6p' thegeekstuff.txt
3. Hardware
4. Security (Firewall, Network, Online Security etc)
5. Storage
6. Cool gadgets and websites

Sed Pattern Format 3: ADDRESS,/PATTERN/

It prints from the Nth line of the input, to the line which matches the pattern. If the pattern doesnt match, it prints upto end of the input.

# sed -n ‘N,/PATTERN/p’ filename

For example, 4th line matches the pattern “Security”, so it prints from 3rd line to 4th line.

# sed -n '3,/Security/p' thegeekstuff.txt
3. Hardware
4. Security (Firewall, Network, Online Security etc)

Sed Pattern Format 4: /PATTERN/,$

It prints from the line matches the given pattern to end of file.

# sed -n ‘/PATTERN/,$p’ filename

# sed -n '/Website/,$p' thegeekstuff.txt
8. Website Design
9. Software Development
10.Windows- Sysadmin, reboot etc.

Sed Pattern Format 5: /PATTERN/,+N

It prints the lines which matches the pattern and next N lines following the matched line.

# sed -n ‘/PATTERN/,+Np’ filename

For example, following prints the 5th line which matches the pattern /Storage/ and next two lines following /Storage/.

# sed -n '/Storage/,+2p' thegeekstuff.txt
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)

Sed Pattern Format 6: /PATTERN/,/PATTERN/

Prints the section of file between two regular expression (including the matched line ).

# sed -n ‘/P1/,/P2/p’ filename

For example, 5th line matches “Storage” and 8th line matches “Design”, so it prints 5th to 8th.

# sed -n '/Storage/,/Design/p' thegeekstuff.txt
5. Storage
6. Cool gadgets and websites
7. Productivity (Too many technologies to explore)
8. Website Design

Delete File Lines Using Address and pattern in Sed

“p” command prints the buffer (Remember to use “-n” option with “p”)

“d” command is just opposite, it’s for deletion. “d” will delete the pattern space buffer and immediately starts the next cycle.

Syntax:

# sed 'ADDRESS'd filename

# sed /PATTERN/d filename

Delete the Nth line.

“Nd” delete the Nth line and prints the other lines.

For example, 3d deletes 3rd line and prints other lines.

$sed 3d sample.txt

Delete Starting from 3rd line and every 2nd line from there.

$sed ‘3~2d’ sample.txt

Delete from 4ht to 8th line from file

$sed ‘4,8d’ sample.txt

Delete the last line from input

$sed ‘$d’ sample.txt

Delete the line which matches the given pattern from input

$sed /Sysadmin/d sample.txt

Delete the line from which matches the given pattern to end of the file

$sed ‘/Website/ ,$d’ sample.txt

Delete the line from which matches the given pattern and 2 lines next to that

$sed ‘Storage/ ,+2d’ sample.txt

Delete blank line from a file using sed.

$sed ‘/^$/d’ sample.txt

Tuesday, 24 August 2010 09:47

Perl Scripting

Written by

Can any one help on these questions which i couldnt answer in my interview,because iam not sure on perl scripting iam learning the things in perl now.

1.What standard Perl modules have you made use of, and why did you use them instead of the equivalent built-in functions?

2.Write a statement that checks to make sure there are at least two command line arguments and exits with an error if not.

3.How do you find the number of elements in an array?

4.Write a regular expression that would match only the last director of a full path filename (and you don't know the path ahead of time).  Example: for "/fee/fi/fo/fum.txt" you would match "fo" with your regular expression

5.Write a Perl script to get the machine host name and print it. No system command should be used; this script should work on Windows and Linux OS without any change.
6.Write a Perl script to check a directory exists or not and print proper message. The directory location will be passed as parameter while executing the script.

Please help me out for the above questions.

 

Saturday, 24 July 2010 05:09

Hudson Master Slave setup

Written by
The tasks can be scheduled to run on the same machine (Master), or on a different machine (Slave). A master is a installation of Hudson, that can manage one or more slaves. The role of master remains same in Master slave setup. It serves HTTP requests, and it can still build projects on it's own.
Slaves are computers that are setup to build projects triggered from the master. A separate program called slave agent runs on slave computer. In this article we'll discuss about how to setup Hudson to executed distributed builds using Master slave. One computer can be configured to execute multiple slave agents.

How it works

When the slaves are registered to a master, the master starts distributing loads to slaves. The delegation depends on the specific job. The job can be configured to either execute on the master, or it can be tied to a specific slave. On the other hand, the jobs can be configured to freely roam between slaves, wherein the job is executed using the free slave. As per the user is concerned, the setup is transparent. The results for all jobs are viewable using the Master, irrespective of the Slave that executed the job.
The slave may be built using any Operating system. The Master slave setup is highly helpful while the user has to execute the job on different Operating system. Consider this use case: The application is expected to run on different operating system, Linux, Solaris and Windows system. To address this need, the user can install Hudson on any machine, say Linux, and add 2 slaves: Solaris and Windows. The user can add 3 different jobs, one running on Master itself and others running on slave machines.

Methods to set up Slave agents

The slave can be launched from Master using any of below methods:
  1. Launch Slave agents on Unix machines via SSH.
  2. Launch Slave agents via JNLP.
  3. Launch Slave agents via execution of command on the master.
  4. Let Hudson control windows slave as a windows service.
The following screenshot illustrates the list of modes under which the Hudson slave can be launched. 


Set up Slave agents on Unix machines via SSH

Hudson has a built-in SSH client implementation that it can use to talk to remote sshd and start a slave agent. This is the most convenient and preferred method for Unix slaves, which normally has sshd out-of-the-box. Click Manage Hudson, then Manage Nodes, then click New Node. In this set up, the connection information is supplied, including SSH host name, user name and authentication credentials. The authentication is performed using password or ssh keys. If it is configured to use ssh keys, the SSH public key should be copied to ~/.ssh/authorized_keys file. Hudson will do the rest of the work by itself, including copying the binary needed for a slave agent, and starting/stopping slaves.

Depending on the project and hardware resource availability, the user Desktop can be used as one of Slave, without affecting his day-to-day activities, thus avoiding the need for dedicated Slaves.

Establish slave agent via Java Web Start

Another way of launching slave is to start a slave agent through Java Web Start (JNLP). In this approach, the user will login to the slave node, open a browser and open the slave page using the URL pointed to the Master. It may look like the following URL:

http://masterserver:port/hudson/jnlpJars/slave.jar
The user is presented with the JNLP launch icon. If user click the icon, the Java Web Start kicks in and it launches a slave agent on this computer.
This mode is convenient when the master cannot initiate a connection to slaves, such as when it runs outside a firewall while the rest of the slaves are in the firewall. The disadvantage is, if the machine with a slave agent goes down, the master has no way of re-launching it on its own.

Set up slave agent headlessly

This launch mode uses a mechanism very similar to Java Web Start, except that it runs without using GUI, making it convenient for an execution as a daemon on Unix. To do this, the user should configure this slave to be a JNLP slave by downloading slave.jar, and then from the slave, run a command like this:
 
java -jar slave.jar -jnlpUrl http://yourserver:port/computer/slave-name/slave-agent.jnlp

The slave.jar file is downloaded from the above mentioned URL. Make sure to replace slave-name with the name of the slave setup in Master.
By default, Hudson runs on port 8080. It can be installed and managed without the need for super user privilege. The super user privilege is not required to manage both Master and Slave.
The below diagram illustrates the list of configuration parameters specific to a slave.

set up Slave Agent using own scripts

If the above modes is not flexible, the user can write his own script to launch the Slave agent. The script is placed in the Master computer and Hudson runs this script whenever it should connect to the slave. The script may use the remote login program like SSH, RSH to establish connection between Master and slave.
The script would execute the slave agent program like java -jar slave.jar. The stdin and stdout for the script should be connected to the master. For example, the script that does ssh myslave java -jar ~/bin/slave.jar would satisfy this need, when it is executed from the Master web interface. For this reason, running this script manually from the command line does no good.

The copy of slave.jar can be downloaded from the above mentioned URL. Launching the slave agent using this mode requires additional setup in the Slave. The benefit is that when the connection goes bad, the user can use Hudson web interface to re-establish the connection.

 

Page 1 of 5