Tag: commands
20 Xargs Commands Examples in Linux / UNIX | Xargs Commands Tutorials
Under Development
20 Mount and Unmount Filesystem / Partition commands in Linux / UNIX
The same -a option can be used with umount to unmount all the filesystems mentioned in /etc/mtab
Example
How to upload the artifacts in Sonatype Nexus | Tutorial
gsh – Run linux commands on many other linux server at once
gsh – Run linux commands on many other linux server at once
Group Shell (also called gsh) is a remote shell multiplexor. It lets you control many remote shells at once in a single shell. Unlike other commands dispatchers, it is interactive, so shells spawned on the remote hosts are persistent.
GSH is a pluggable version of DSH (Distributed Shell) written in Python. Both a module and a command-line tool for running a shell command over multiple machines are included. GSH can be extended by adding new host loaders as well as hooking into various stages of the runtime.
It requires only a SSH server on the remote hosts, or some other way to open a remote shell.
gsh allows you to run commands on multiple hosts by adding tags to the gsh command.
e.g > gsh tag “remote command”
Important things to remember:
/etc/ghosts contains a list of all the servers and tags. gsh is a lot more fun once you’ve set up ssh keys to your servers
gsh [OPTIONS] SYSTEMS CMD…
SYSTEMS is a combination of ghost macros. See ghosts(1).
CMD is the command to run
-h, –help Display full help
-d, –debug Turn on exeuction debugging reports
-h, –no-host-prefix Does not prefix output lines with the host name
-s, –show-commands Displays the command before the output report
-n, –open-stdin Leaves stdin open when running (scary!)
-l, –user USER SSH’s to the host as user USER
-r, –run-locally Run commands locally (replaces $host with host)
-o, –self-remote Run locally instead of over SSH for local host
-V, –version Report the version and exit
You set up a /etc/ghosts file containing your servers in groups such as web, db, RHEL4, x86_64, or whatever (man ghosts) then you use that group when you call gsh.
How to run the gsh?
> gsh linux “cat /etc/redhat-release; uname -r”
You can also combine or split ghost groups, using cpanel+vps or web-RHEL4, for example.
Here’s an example /etc/ghosts file:
# Machines # # hostname OS-Version Hardware OS cp security linuxbrigade.com debian6 baremetal linux plesk iptables linuxbrigade.com centos5 vps linux cpanel csfcluster linuxbrigade.com debian7 baremetal linux plesk iptables linuxbrigade.com centos6 vps linux cpanel csfcluster linuxbrigade.com centos6 vps linux cpanel csfcluster linuxbrigade.com centos6 vps linux nocp denyhosts linuxbrigade.com debian6 baremetal linux plesk iptables linuxbrigade.com centos6 baremetal linux cpanel csf linuxbrigade.com centos5 vps linux cpanel csf
Reference url
http://www.linuxbrigade.com/run-the-same-command-on-many-linux-servers-at-once/
http://linux.die.net/man/1/gsh
https://github.com/gmjosack/gsh
http://outflux.net/unix/software/gsh/
http://outflux.net/unix/software/gsh/gsh.html
Linux Commands for Administrator | Linux Admin Commands Reference
su: Acquiring Superuser Status
# su :- Acquire Superuser after entering password but home directory does not change
# su – l :– Acquire Superuser access after password but home directory change to ROOT
# su – henry : – To Login and Recreate a user environment
# sudo -s root -c “/bin/bash -l”
passwd – Changing any Password
# passwd henry
stat <fileName>
ulimit – Setting Limits on file size
ulimit command impost a restriction on the maximum size of a file that a user is permitted to create
# ulimit 20971510 This is often placed in /etc/profile. Measured in 512 byte blocks
umask – Please refer user manual
Source
The source command in shell is used to execute commands from a file in the current shell. This is useful to load function or variables stored in another file.
# source filename
# source /path/to/file
locate
To locate any files in system. This works based on updatedb.
# locate
# updatedb Run as a root.File location is /var/lib/mlocate/mlocate.db
which
shows the full path of (shell) commands
od
od command dumps a file in octal, decimal, and other formats.
# od -c special-chars.txt
# od -bc special-chars.txt
chkconfig
chkconfig command is used to setup, view, or change services that are configured to start automatically during the system startup.
The –list option displays all the services with the current startup configuration status.
# chkconfig –list
To view only the services that are configured to be started during system startup, do the following. Please note that this assumes that your system startup level is 3
# chkconfig –list | grep 3:on
To display the current settings for a selected service only, use chkconfig –list followed by the name of the service:
# chkconfig –list service_name
To enable a service in runlevels 2, 3, 4, and 5, type the following at a shell prompt as root.
# chkconfig service_name on
To disable a service in runlevels 2, 3, 4, and 5, type the following at a shell prompt as root.
# chkconfig service_name off
# chkconfig network –level 3 && echo “Network service is configured for level 3”
# chkconfig network –level 1 && echo “Network service is configured for level 1”
To view the startup configuration of a particular service, grep the output of ‘chkconfig –list’ for that service.
# chkconfig –list | grep network
Add a new Service to the Startup
# chkconfig –add iptables
To remove it from the startup list, use the –del option as shown below.
# chkconfig –del ip6tables
How to install services?
Step 1. Copy the script into /etc/init.d/ such as /etc/init.d/httpd
Step 2. chkconfig --add httpd
http://bitnami.org/article/how-to-install-services-on-linux
service
# service network restart
OR
# /etc/init.d/network restart
To start Linux network service:
# service network start
To stop Linux network service:
# service network stop
date – Setting the system Date
# date MMDDhhmm
groupadd
# groupadd –g 241 dba The command place entry in /etc/group which can also insert manually(dba:x:241). 241 is GID of dba
useradd
# useradd –u 210 –g dba –c “The RDBMS” –d /home/oracle –s /bin/bash –m username
usermod
# usermod –s /bin/csh username
userdel
# userdel username
shutdown
# shutdown 17:30 Shutdown at 17:30
# shutdown –r now Shutdown immediately and reboot
# shutown –r 17:30 Shutdown immediately and reboot at 17:30
# shutown –h now Shutdown immediately and halt
# shutown –h 17:30 Shutdown at 17:30
# shutdown –c Cancel the shutdown
Reboot / Poweroff / halt
df
The df (disk free) command lets you see if file systems are full.
- To see how much free space is in all file systems (including mounted ones), use df
- To see how much free space is in all file systems with names containing “dev”, use df | grep dev
- To see how much free space is in your home file system, use df /home
- To see how much free space is in the file system “tmp”, use df /tmp
- To see if there is enough free space on the machine, check the output from the following
# df /usr,
# df /var
# df -k /tmp /home -k Reports in units of KB
# df –h / /usr -h reports in MB, GB
du -Disk usage
# du /home/sales/tml
# du –s /home/* -s options for summary disk usage information for tml directory
# du –sh /home -h options for reports in MB , GB
# find /home –size +1024 –print Files above 1MB
# find /home –size +1024 –size -4096 – print Files above 1Mb and below 4MB
mount / unmount
Using file for incremental Backup
# tar –cvf /dev/rct0 `find /home –type f –newer .last_time –print`
# touch .last_time
# tar –cvf /dev/rct0 /home > homelist `date “+%d%b%y”`
xrags
xargs vs. exec {}
Remote machine Login: telnet, rlogin
telnet
rlogin
Secure Login with password: ssh
ssh
Remote login without Password: rsh
rsh
File Transfer: ftp, rcp, scp
rcp
scp
ftp
ftp rajesh-servers – How to Login
Directory Commands:
# cd [directory] Changes to the specified directory on the remote system.
# dir Displays a long listing of files on the remote system.
# ls Displays a short listing of the files on the remote system.
# lcd Changes the working directory on the local system.
# pwd Displays the working directory on the remote system.
File Transfer Commands:
# get [filename] Downloads a single file from the remote system.
# put [filename] Uploads a single file to the remote system.
# mget [file…] Downloads multiple files from the remote system.
# mput [file…] Uploads multiple files to the remote system.
Generic troubleshooting tips
- Use tail –f to watch log file in real time, advantage is simple you can spot error or warning message in real time.
tail –f /path/to/log/file
Example(s):
# tail –f /var/log/maillog - Use telnet command to see if you get response or not. Sometime you will also see some informative message:
telnet ip port
Example(s):
# telnet localhost 53
# telnet localhost 25
- Make sure you can see PID of your service.
pidof service-name
cat /var/run/service.pid
Example(s):
# pidof sshd
# cat /var/run/sshd.pid - You need to make sure that your DNS server or third party DNS server (ISP) is accessible. This is an important step, as many network services depend upon DNS; especially sendmail/postfix or Squid etc for example. Run dig or nslookup. No timeout should occurred.
# dig your-domain.com
# nslookup gw.isp.com
# more /etc/resolv.conf
Performance / Memory / CPU / Process check
ps – Read user manual
w username – Find Out Who Is Logged on And What They Are Doing. w command displays information about the users currently on the machine, and their processes.
top
The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.
uptime
tell How Long The System Has Been Running
free – Memory Usage
The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free –m Display free memory size in MB
# free -t –m Displays a line containing the totals memory in MB
vmstat
Linux Performance Monitoring using iostat, mpstat and vmstat
iostat
pmap
mpstat
# strace & truss – Debug the Execution of a Program in Linux
Trace the Execution of an Executable
# strace ls
Save the Trace Execution to a File Using Option -o
strace -o output.txt ls
Execute Strace on a Running Linux Process Using Option -p
# ps -C firefox-bin To get the process id
# sudo strace -p 1725 -o firefox_trace.txt
Print Relative Time for System Calls Using Option -r
# strace -r ls
Networking troubleshooting tips
Hostname verification or setup tools
- hostname : To get hostname of server.
- hostname –s : To get FQDN hostname of server
- more /etc/sysconfig/network : To setup hostname and networking can enable or disabled.
- dnsdomainname : List or setup domainname.
- more /etc/hosts : Make sure at least localhost entry do exist.
Ethernet configuration tools
service network reload|restart|stop|start : To reload (after changed made in ip config file)|restart|stop|start network interface with all properties.
route
The route command allows you to add static routes to the routing tabling.
traceroute (tracert in Windows)
Traceroute displays the routers that are passed through to reach the destination.
dmesg
When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process.
These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten. You could see all those messages after the system bootup using the dmesg command.
# dmesg | more
# dmesg | grep Memory
# dmesg | grep eth
ifconfig
The ifconfig command is used to display the local interface configuration (winipcfg for Windows) and to modify the configuration.
ifconfig eth0 up|down : To enable|disable network interface
netstat
The network status command netstat displays status information about the network interfaces on the host machine. Netstat command displays the network related information such as network connections, routing tables, interface statistics.
-a Show both listening and non-listening sockets
-n prints the numeric forms of IP address
# netstat -ain Local interface status
# netstat -rn Routing table information
# netstat –an Display Active Internet Connections and domain sockets using netstat
# netstat -tap Display Active Connections with Process ID and Program Name
# netstat –-route Display Routing Table
# netstat –statistics –raw Display RAW network statistics
# netstat –tcp –numeric List of TCP connection to and from the machine.
netstat -tuplin UNIX
iptables
To know which ports are running in the machine.
ping
The ping command allows you to determine that the:
- TCP/IP stack is configured properly
- Network interface card is configured properly
- Default gateway and subnet mask is configured properly
- Domain name services is configured properly.
# ping ipaddress to check if the ip address in connected with network and if host is alive or dead
# ping 127.0.0.1/localhost – to check TCP/IP services is running properly or not. Checks that your TCP/IP stack is working properly up to the NIC. If this fails, check to see if you have TCP/IP services loaded.
# ping “IP address of default gateway” – Checks that the NIC is working on the local subnet by pinging the local side of the default gateway which is a router. If this fails, check that TCP/IP is bound to the NIC. Then check that the NIC’s IRQ, and base address are set properly both on the card itself and in the operating system’s interface configuration.
# ping “domain name” if DNS name is correctly configured. If it fails, check that the DNS server’s IP address is entered in properly in the TCP/IP configuration.
arp
address resolution protocol
# arp – To see the IP to MAC address translation table
# arp -a -Use this command to see the IP to MAC address translation table if you are having problems connecting to other network hosts. It is a dynamic cache which updates every 120 seconds.
# more /etc/modules.conf : To see your network card configuration alias for eth0 exists or not.
# lsmod : To list loaded modules (read as drivers), here you need to see that eth0 module is loaded or not, if not loaded then use insmod to insert (load) driver.
# dhclient : Dynamic Host Configuration Protocol Client, run this if your Ethernet card is not getting ip from DHCP box on startup; this command does by default shows useful information.
To see if service blocked because of access control
# iptables –n –L : To list all iptable rules; useful to see if firewall blocks service or not.
# service iptables stop|start : To start|stop iptables
# more /etc/xinetd.conf
sar
sar is an excellent monitoring tool that displays performance data of pretty much every resource of the system including CPU, memory, IO, paging, networking, interrupts etc., Sar Collects, Reports (displays) and Saves the performance data.
# sar –u | more Display CPU Statistics using Sar Command
# sar –d | more Display Disk IO Statistics using sar command
# sar -n DEV | more Display networking Statistics using sar command
# sar -n SOCK |more Display networking Statistics using sar command
# sar -n DEV -f /var/log/sa/sa24 | more To display the network counters from the 24th
# sar 4 5 You can also display real time usage using sar
lsof
lsof stands for “ls open files”, which will list all the open files in the system. The open files include network connection, devices and directories. The output of the lsof command will have the following columns:
COMMAND process name.
PID process ID
USER Username
FD file descriptor
TYPE node type of the file
DEVICE device number
SIZE file size
NODE node number
NAME full path of the file name.
# lsof | more View all open files of the system
# lsof | wc -l How many files are open in the system at any given point
# lsof –u ramesh View open files by a specific user. A system administrator can use this command to get some idea on what users are executing on the system.
# lsof /bin/vi it displays all users who are currently using vi
Some external Tools
iptraf – Real-time Network Statistics
tcpdump – Detailed Network Traffic Analysis
Nagios – Server And Network Monitoring
Cacti – Web-based Monitoring Tool
KDE System Guard – Real-time Systems Reporting and Graphing
Gnome System Monitor – Real-time Systems Reporting and Graphing
nmap – scan your server for open ports.
lsof – list open files, network connections and much more.
ntop web based tool – ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
Conky – Another good monitoring tool for the X Window System. It is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage,
temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes etc.
GKrellM – It can be used to monitor the status of CPUs, main memory, hard disks, network interfaces, local and remote mailboxes, and many other things.
vnstat – vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).
htop – htop is an enhanced version of top, the interactive process viewer, which can display the list of processes in a tree form.
mtr – mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.
yum
Rsync
rpm – Refer Cheat Book
Some Well known System services
QUICK DNS Flush /etc/rc.d/init.d/nscd restart
/Proc file system – Various Kernel Statistics
# cat /proc/cpuinfo
# cat /proc/meminfo
# cat /proc/zoneinfo
# cat /proc/mounts
Questions
What is MASK in output of ifconfig
Understand the output of ifconfig in linux such as RX packets and TX packets
Set ip address using commands line
Change Host name in RHEL
Changes should be made in the following files.
/etc/hosts
/etc/sysconfig/network
For finding your java process
jps -v For finding your java process.
jstack PID To Get stack of threads inside jvm
If application running under different user than yourself
sudo jps -v
sudo jstack 6172
Linux Commands for User | Linux User Commands Reference
General Commands Not Used Frequently
cal – The Calender
bc – The Calculator
file – knowing the file types
cmp – Comparing two files.
comm – Common in two files
dos2unix and unix2dos – Converting between dos and unix
stat – Display file or file system status such as Access, Modify,Change, users and more
# stat <fileName> unix Command
General Commands Frequently Used
Script – recording your session
Passwd – change your password
Echo – Displaying the message
Printf – Displaying the message
who – Who are the Users
who am I – To know my user name
uname – Knowing your machine characteristics
# uname –a – To Print all info.
tty – knowing your terminal
pwd – Checking Current Directory
cd – changing the current Directory
mkdir – Making directories
rmdir – removing directories
ls – Listing Directory Contents
# ls –F – Marks executables with *, directories with / and symbolic links with @ at the end of file name.
# ls – a – Shows all filesnames including beginning with a dot.
# ls –R – Recursive display list of files and folders
# ls -1 – one filename in each line
# ls –l long listing + time of last file modification
# ls –t Sorts filenames by last modification time
# ls –u – Sorts filenames by last access time
# ls –I – Displays inode number
# ls –lc – Time of last inode modifcation
Cat – Displaying and creating Files
# cat filename – Displaying file contents
# cat –n filename – Displaying file contents with line numbers
# cat > filename – To Create a file name
More – Paging Output
# more filename – To to display the filename contents
Less – Paging Output
wc – Counting lines, Words and Characters
# -l – Option counts only the number of lines
# -w – Option counts only the words.
# -c – Options counts only the characters.
cp – copying a files
Options
# -p – Copy the files including preserve the mode, ownership and permissions.
# -R – Copying Directory Structure
gzip and gunzip – Compressing and Decompressing Files
# gzip filename
# gzip –l – To find out how much of the compression was archived.
# gzip –d filename – To Decompress a file
# gunzip filename – To Decompress a file
# gzip –R dir – To Compress Recursive
Tar – The archival Program
# -c – To create a archive
# -x – Extract files from archive
# -t – Display fils in archive
# -v – verbose
# -f – name of the arch filename .tar
# tar –cvf filename.tar filename1 filename2
# tar –xvf filename.tar
Create tar and gzip
# tar cvf – foodir | gzip > foo.tar.gz
Un-tar and gunzip
# tar -zxvf apache-activemq-5.1.0-bin.tar.gz
Un-tar and uncompress using gunzip
# gunzip -c apache-activemq-5.1.0-bin.tar.gz | tar xvf –
or alternatively:
# gunzip apache-activemq-5.1.0-bin.tar.gz | tar xvf –
File Attributes
chmod – Changing File Permission
Abbreviations Used by chmod
Category | Operation | Permission | Assigned Number |
u -user | + – Assigns Permission | r – Read permission | 4 |
g -group | – – Assigns Permission | w – Write Permission | 2 |
o – other | = – Assigns Permission | x – Execute Permission | 1 |
a –all(ugo) |
Example:
# chmod u+x filename
# chmod u+x filename
# chmod ugo+x filename
# chmod u+x filename1 filename2 filename3
# chmod a-x,go+r filename
# chmod 566 filename
# chmod 755 filename
# chmod –R 755 .
# chmod –R a+x *
Chown – Changing File Ownership
# chown username filename
More example needed.
Shell’s Wild-Cards
Wild-Card | Matches |
* | Any numbers of characters including none |
? | A single character |
[ijk] | A single character – either I,j or k |
[x-z] | A single character that is within the ASCII range of the characters x and z |
[!ijk] | A single character that is not an I,j, or k |
[!x-z] | A single character that is not within the ASCII range of the characters x and z |
Ps – Process Status
Options to ps
POSIX Options | Significance |
-f | Full listing showing the PPID of each process |
-e or –A | All Process including user and Systems |
-u usr | Process of user usr only |
-a | Processes of all users excluding processes not associated with terminal |
-l | A long listing showing memory-related information |
Example
# ps -ef
# ps –u sumit
# ps –a
# ps –e
# ps -A – ps command will report a snapshot of the current processes. To select all processes use the -A
# ps -Al – Show Long Format Output
# ps -AlF – To turn on extra full mode (it will show command line arguments passed to process):
# ps axu – Print All Process On The Server
# ps -U vivek -u vivek u – See Every Process Running As User Vivek
# ps -p 55977 -o comm= – Display The Name of PID 55977
# ps -auxf | sort -nr -k 4 | head -10 – Find Out The Top 10 Memory Consuming Process
# ps -auxf | sort -nr -k 3 | head -10 – Find Out top 10 CPU Consuming Process
# $! – Store the PID of the last background jobs
# echo $$ – To Know the PID of current Shell
# echo $SHELL – To Know the Current Shell
# System process easily identifies the ? in the TTY coloumn.
# Mechanism of Process Creation
Fork
Exec
Wait
& and nohup – Ruunning jobs in background
# nohup sort emp.lst $
nice – Job execution with low priority
Kernel decides how much processor time is required for a process based on the nice value. Possible nice value range is: -20 to 20. A process that has a nice value of -20 is very high priority. The process that has a nice value of 20 is very low priority.
Use ps axl to display the nice value of all running process as shown below.
# ps axl
# nice command is used with & operator to reduce the priority of jobs
# nice script.sh &
# ./nice-test.sh & Default the nice value of 0
# nice -10 ./nice-test.sh & Nice value is 10 Low priority.
# nice –10 ./nice-test.sh & Nice value is -10 High priority.
Kill – killing the process
# kill PID
# kill PID1 PID2 PID3
# kill $i – Killing the last background process
# kill –s KILL PID – recommended way for killing (SIGKILL)
# kill -9 PID – same as above but not recommended. (SIGKILL)
# kill -9 $$ -$$ store the PID of current shell
# kill –s KILL 0 – kill all the process including the login shell.
# kill –l – To View the list of all signal names and numbers that are available on your machine.
# kill %1 – Kill first background jobs
Jobs
List the background jobs in following fashion
[3] + running command
[1] – running command
[2] running command
bg – Convert Jobs to background
# if you have invoked a command and the prompt has not yet return, you can suspend the job by pressing Ctrl-Z. Observe that job has not been terminated yet; its onl y suspended (“stopped”). Now you can use bg command to push the current foreground job in the background.
# bg %2 – Sends second job to background
# bg %sort – Sends sort job to background
fg – bring background job to foreground
# fg – To bring most recent job to forground
# fg %1 – Bring First job to foreground
# fg %2 – Bring second job to foreground
# fg %sort – Bring sort job to foreground
at – On time execution
# at 14:08
at > script.sh
[ctrl-d]
Batch – execute when system resources are available
# batch < script.sh
Cron – Schedule and run jobs periodically
# crontab –e – To Edit the cron tab
# crontab –l – To Display the cron tab
# crontab cron.tx – cron.txt contains cron commands
# crontab –r – To Remove the cron
Format of crontab – TODO
Customizing the environment
# set – set statemtent display a complete list of all environment variable
# PATH =$PATH:/usr/xpg4/bin – Adding new value to old values
# PS1 =”C> “ – To Change the prompt
# PS1=’[$PWD] ‘ – To Change the prompt to pwd
# alias cp=”cp –I” – To Set the alias in bash
# history – To See the history
# IFS – Field Separators for commands and arguments
# !! – Repeat Previous commands
# !2 – Repeat commands 2 from history output
# !-2 – Execute the commands prior to previous one
# !v – Execture very last commands beginning withg v
# $_ – Using last arugement of previous commands
# mkdir raj
# cd $_
ln – Hard Links and Softlinks
# ln /usr/bin/perl /usr/local/bin/perl – To create a hard links from src to dest
# ls –I – To Display the node number of files
# ln –s /usr/bin/perl /usr/local/bin/perl – To Create a soft link
umask – default file and directory permission
# When you create a files and directories, the permission assigned to them depends on rge system’s default setting..
# unix has default 666 for regular files & 777 for directories
# umask –ENTER
# 022
This is an octal number which has to be subtracted from system default to obtain the actual default. This becomes 644 (666-022) for ordinary files and 755 ( 777 -022) for directories.
# User can set the umask such as umask 023.
touch – Changing the time stamp
# touch emp.lst – Create a file name called emp.lst
# touch 02161430 emp.lst – To Change the time stamp of file for MMDDhhmm format
# touch –a 02161430 emp.lst – To Change the access time stamp of file for MMDDhhmm format
# touch –m 02161430 emp.lst – To Change the modification time stamp of file for MMDDhhmm format
find – Locating files
Expression Used by find.
Expression | Use |
-inum n | Having inode number n |
-type x | if of type x can include files, directories or symbolic link |
-type f | If an ordinary file |
-perm nnn | If octal permission match nnn completely |
-links n | If having n links |
-user usname | If owner by usname |
-group gname | If owned by group gname |
-size +x[C] | |
-mtime –x | If modified in less than x days |
-newer filename | If modified after filename |
-mmin –x | If modified in less than x minutes |
-atime +x | If accessed in more than x days |
-amin +x | If accessed in more than x minutes |
-name filename | Filename |
Action | Significance |
Prints selected file on standard output | |
-ls | Executes ls –lids commands on selected files |
-exec cmd | Executes UNIX command cmd followed by {} \; |
-ok cmd | Like –exec, except that command is executed after user confirmation |
Example
Change File Permissions Recursively
# find . -type f -exec chmod 644 {} \;
# find . -type d -exec chmod 755 {} \;
# find . -name Configuration.php -exec chmod 666 {} \;
Find files modified in the last 48 hours, and in current folder and one level below
# find -maxdepth 2 -type f -mtime -2
To find all files modified in the last 24 hours (last full day) in a particular specific directory and its sub-directories:
# find /directory_path -mtime -1 –print
To find all files with regular file types only, and modified in the last 24 hours (last full day) in current directory and its sub-directories:
# find /directory_path -type f -mtime -1 –print
# find . -type f -mtime -1 –print
To find all files that are modified today only (since start of day only, i.e. 12 am), in current directory and its sub-directories:
# touch -t `date +%m%d0000` /tmp/$$
# find /tmefndr/oravl01 -type f -newer /tmp/$$
# rm /tmp/$$
To find all files in /home/user/demo directory
# find /home/user/demo -type f –print
To find all files in /home/user/demo directory with permission 777, enter:
# find /home/user/demo -type f -perm 777 –print
Apply new permission using the -exec option as follows:
# find /home/user/demo -type f -perm 777 -print -exec chmod 755 {} \;
To select directories and subdirectories use the following syntax:
# find /var/www/html -type d -perm 777 -print -exec chmod 755 {} \;
This first Linux find example searches through the root filesystem (“/”) for the file named “Chapter1”. If it finds the file, it prints the location to the screen.
# find / -name Chapter1 -type f –print
A nice thing to know is that on Linux systems and modern Unix system you no longer need the -print option at the end of the find command, so you can issue it like this:
# find / -name Chapter1 -type f
This next find command searches through the /usr and /home directories for the file named Chapter1:
# find /usr /home -name Chapter1 -type f
To search in the current directory, and all subdirectories, just use the . character to reference the current directory in your find commands, like this:
# find . -name Chapter1 -type f
This next command searches through the /usr directory for all files that begin with the letters Chapter, followed by anything else. The filename can end with any other combination of characters. It will match filenames such as Chapter, Chapter1, Chapter1.bad, Chapter-in-life, etc.:
# find /usr -name “Chapter*” -type f
This next command searches through the /usr/local directory for files that end with the extension .html. These file locations are then printed to the screen.
# find /usr/local -name “*.html” -type f
To find all directories named build under the current directory, use this command:
# find . -type d -name build
This command searches through the htdocs and cgi-bin directories for files that end with the extension .cgi. When these files are found, their permission is changed to mode 755 (rwxr-xr-x). This example shows that the find command can easily search through multiple sub-directories (htdocs, cgi-bin) at one time.
# find htdocs cgi-bin -name “*.cgi” -type f -exec chmod 755 {} \;
Find and display files last modified les than 90 days ago.
# find . -name “*” -mtime -3 -print
find everything in your home that has been modified more recently than “abc.txt”:
# find $HOME -newer ~joeuser/lastbatch.txt
For finding only files from all directories recursively
# find ./ -type f | wc -l
For finding only files from all directories recursively
# find ./ – type d | wc -l
How to Return a message when a file is not found using find command? OR
Find command return type OR
when the file is not found i want it to return some value OR
# find . -name raj.txt > raj.txt
# counter= `cat abc.txt | wc -l’
# if [$counter -gt 0]
# then
# echo “File is found”
# else
# echo “File is not found”
# fi
That’s nice, but what if I want to see the last modification time of these files, or their filesize? No problem, I just add the “ls -ld” command to my find command, like this:
find . -name “*.pl” -exec ls -ld {} \;
Count Total number of files in Directory and Subdirectory
# find . -type f | wc –l
Count Specific extention files in Directory and Subdirectory
# find . -type f -name \*.mnp |wc –l
Count only Directory
# find . -type d | wc –l
head – Displaying the beginning of a file
# head –n 3 filename
# vi `ls –t` | head –n 1` – Opens last modified file for editing
tail – Displaying the end of a file
# tail -3 filename
# tail –f filename Monitering file live
Cut
# cut –c 6-12,24-32 filename – Cutting column
# cut –d \| -f 2,3 filename – Cutting fields
# cut –d “|” –f 1,4- filename –To cut out the fields numbered 1,4,5 and 6.
Sort – Ordering a file
Sort Options
Option | Description |
-tchar | Use delimiter char to identify fields |
-k n | Sorts on nth field |
-k m,n | Start sort on mth field and end sort on nth field |
-k m.n | Start sort on nth column of mth field |
-u | Removes repeated lines |
-n | Sort numerically |
-r | Reverse sort order |
-m list | Merge sorted files in list |
-c | Checks of file sorted |
-o filename | Place output in file filename |
Examples:
# sort –t”|” –k 2 shortlist
# sort –t”|” –r –k 2 shortlist
# sort –t “|” –k 3,3 –k 2,2 shortlist
# sort –t”|” –k 5.7,5.8 shortlist
# sort numfile
# cut –d”|” –f3 filename | sort –u
# sort –m foo1 foo2 foo3
Uniq – Locate repeated and non-repeated lines
# sort dept.lst | uniq
# cut –d”|” –f3 emp.lst | sort | uniq –u ( -u selects only lines which is not repeated)
# cut –d”|” –f3 emp.lst | sort | uniq –d ( -d selects one copy of repeated lines)
# cut –d”|” –f3 emp.lst | sort | uniq –c
tr – Translating characters
# tr ‘|/’ ‘~-‘ < emp.lst | head –n 3 – To replace the | with a ~ and the / with a -.
# head –n 3 emp.lst | tr ‘[a-z]’ ‘[A-Z]’ – Change first three lines from lower to upper.
# tr –d ‘|’ < emp.lst | head –n 3 – To deleting characters
grep – Searching for a pattern
# grep “director” filename1 filename2
# grep “Rajesh Kumar” filename1
# grep –i ‘rajesh’ filename1 To Ignore case
# grep –v ‘rajesh’ filename1 > filename2 To Select all except lines containing patterns
# grep –n ‘rajesh’ filename1 To Display line numbers
# grep –c ‘rajesh’ filename Counting line containing pattern
# grep –l ‘rajesh’ *.lst -l options display only the name of files containing pattern
# grep –f pattern.lst emp.lst Taking patterns from files
Grep a file, but show several surrounding lines?
For BSD or GNU grep you can use -B num to set how many lines before the match and -A num for th number of lines after the match.
grep -B 3 -A 2 foo README.txt
If you want the same amount of lines before and after you can use -C num.
grep -C 3 foo README.txt
This will show 3 lines before and 3 lines after.
Basic Regular Expression tables
Symbols or Expressions | Matches |
* | Zero or more occurrances of the previous character |
g* | Nothing or g,gg,ggg, etc |
. | A Single Character |
.* | Nothing or any number of character |
[pqr] | A single character p, q or r |
[c1-c2] | A Single Character with ASCII range |
[1-3] | A single digit between 1 and 3 |
[^pqr] | A Single character which is not a p,q or r |
[^a-zA-Z] | A non-alphabetic character |
^pat | Pattern pat at the beginning of the line |
pat$ | Pattern pat at the end of the line |
bash$ | Bash at the end of the line |
^bash$ | Bash as the only one word in line |
^$ | Line containing nothing |
+ | Matches one or more character of previous character |
? | Matches zero or one occurrence of the previous character |
| | Delimiter for multiple pattern |
( ) | Group pattern |
Example:
# grep “[aA]g[ar][ar]wal filename
# grep “[aA]gg*[ar][ar]wal” filename
# grep “j.*Saxena” filename
# grep “^2” filename
# grep “7…$” filename
# grep “^[^2]” filename
# grep –E “[aA]gg?arwal” filename
# grep –E ‘sengupta|dasgupta’ filename
# grep –E ‘(sen|das)gupta’ filename
Editor
Awk – http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/
Sed
vi
# sed – The Stream Editor
# sed ‘3q’ filename == head –n 3 Quit after line number 3
# sed –n ‘1,2p’ filename Prints the first 2 lines. Must use –n with p
# sed –n ‘$p’ filename print last line
# sed –n ‘9,11p 7,9p $p’ filename
# sed –n ‘3,$ip’ filename Don’t print line 3 to the end, display only line 1 and 2
# sed –n ‘/director/p’ filename To print the lines which has pattern in filename
# sed –n ‘/dasgupta/,/saksena/p’ filename
# sed –n ‘1,/dasgupta/p’ filename
# sed –n ‘/[aA]gg*[ar][ar]wal/p’ filename
# sed –n ‘/sa[kx]s*ena/p /Gupta/p’ filename
# sed –n ‘/50…..$/p’ filename
# sed –n ‘director/w dlsit’ filename
# sed –n ‘director/w dlist /manager/w mlist /executive/w elist’ filename
# sed –n ‘1,500w foo1 501,$w foo2’ filename
# sed ‘1i\
> #include <stdio.h>\
> #include<unistd.h>
>’ foo.c >> $$ Include these include on beginning of the program
# sed ‘a\
‘ filename # insert after every line this blank line
# sed “/director/d’ filename > filename2 -n option not to be used with d
== # grep –v “director” filename > filename2
# sed –n ‘/director/!p’ filename1 > filename2
# sed ‘s/|/:/’ filename | head -2
# sed ‘s/|/:/g’ filename | head -2
# sed ‘1,3s/|/:/g’ filename
# sed ‘1,5s/director/member /’ filename
# sed ‘s/^/2’ filename
# sed ‘s/$.00/’ filename
Sed tables
Command | Description |
I,a,c | Insert, Appends and Changes text |
d | Delete lines |
10q | Quit after reading the first 10 lines |
p | Print line on standard outputs |
3,$p | Print lines 3 to the end. –n option is required |
$!p | Prints all lines except last line. –n option required |
/begin/,/end/p | Print lines enclosed between begin and end. –n option required |
q | Quit after reading uo to the address line |
Vi – Editor
IMPORTANT – YOU SHOULD USE DOUBLE QUOTES ONLY WHEN PARAMETER EVALULATION OR command substitution is embedded within command
Write about command “w”
Perforce backup commands | Perforce backup method
There are several commands and arguments that allow more complex operations. (A command line flag “-z” tells Perforce to use compre ssion.) The commands are as follows:
p4d -jc [-z] – Take a checkpoint, rotate the journal
p4 admin checkpoint [-z] – Client-side checkpoint/journal
p4d -jj [-z] – Rotate the journal
p4d -jr [-z]filename – Restore the journal file filename
p4d -jd [-z] filename – Checkpoint to filename without incrementing the checkpoint/journal counter
p4d -c command – Lock the database files, and run command
How to Execute external commands by using perl?
There are many ways to execute external commands from Perl. The most commons are:
- system function
- exec function
- backticks (“) operator
- open function
All of these methods have different behaviour, so you should choose which one to use depending of your particular need. In brief, these are the recommendations:
method | use if … |
system() | you want to execute a command and don’t want to capture its output |
exec | you don’t want to return to the calling perl script |
backticks | you want to capture the output of the command |
open | you want to pipe the command (as input or output) to your script |
More detailed explanations of each method follows:
Using system()
system() executes the command specified. It doesn’t capture the output of the command.
system() accepts as argument either a scalar or an array. If the argument is a scalar, system() uses a shell to execute the command (“/bin/sh -c command”); if the argument is an array it executes the command directly, considering the first element of the array as the command name and the remaining array elements as arguments to the command to be executed.
For that reason, it’s highly recommended for efficiency and safety reasons (specially if you’re running a cgi script) that you use an array to pass arguments to system()
Example: #-- calling 'command' with arguments system("command arg1 arg2 arg3"); #-- better way of calling the same command system("command", "arg1", "arg2", "arg3"); The return value is set in $?; this value is the exit status of the command as returned by the 'wait' call; to get the real exit status of the command you have to shift right by 8 the value of $? ($? >> 8). If the value of $? is -1, then the command failed to execute, in that case you may check the value of $! for the reason of the failure.
Example: system("command", "arg1"); if ( $? == -1 ) { print "command failed: $!\n"; } else { printf "command exited with value %d", $? >> 8; }
Using exec()
The exec() function executes the command specified and never returns to the calling program, except in the case of failure because the specified command does not exist AND the exec argument is an array.
Like in system(), is recommended to pass the arguments of the functions as an array.
Using backticks (“)
In this case the command to be executed is surrounded by backticks. The command is executed and the output of the command is returned to the calling script.
In scalar context it returns a single (possibly multiline) string, in list context it returns a list of lines or an empty list if the command failed.
The exit status of the executed command is stored in $? (see system() above for details).
Example: #-- scalar context $result = `command arg1 arg2`; #-- the same command in list context @result = `command arg2 arg2`; Notice that the only output captured is STDOUT, to collect messages sent to STDERR you should redirect STDERR to STDOUT
Example: #-- capture STDERR as well as STDOUT $result = `command 2>&1`;
Using open()
Use open() when you want to:
– capture the data of a command (syntax: open(“command |”))
– feed an external command with data generated from the Perl script (syntax: open(“| command”))
Examples: #-- list the processes running on your system open(PS,"ps -e -o pid,stime,args |") || die "Failed: $!\n"; while ( <PS> ) { #-- do something here } #-- send an email to user@localhost open(MAIL, "| /bin/mailx -s test user\@localhost ") || die "mailx failed: $!\n"; print MAIL "This is a test message";
Common unix commands and utilities | unix commands and utilities Guide
To find a program not on this list (and there are hundreds), try the man program with the -k option followed by a keyword, for example,
man -k tape
to find all on-line manual entries that refer to programs that work with tapes.
Terminal control characters for C-shell (csh)
^h, backspace | erase previously typed character |
^u | erase entire line of input so far typed |
^d | end-of-input for programs reading from terminal |
^s | suspend writing to terminal (freezes terminal) |
^q | continue writing to terminal |
^z | suspend currently running job; restart with bg or fg |
^c | kill currently running program and allow clean-up before exiting |
^\ | emergency kill of currently running program with no chance of cleanup |
Also see a list of special characters that should not be used in filenames.
Login and authentication
login | access computer; start interactive session |
logout | disconnect terminal session |
passwd | change local login password; you must set a strong password that is not easily guessed |
kinit | obtain kerberos ticket for connections to other kerberized computers |
kdestroy | destroy kerberos tickets (authorizations) |
Information
date | show date and time |
history | list of previously executed commands |
man | show online documentation by program name |
info | online documentation for GNU programs |
w, who | who is on the system and what they are doing |
whoami | who is logged onto this terminal |
top | show system stats and top CPU using processes |
uptime | show one line summary of system status |
File management
cat | combine files |
cp | copy files |
ls | list files in a directory and their attributes |
mv | change file name or directory location |
rm | remove files |
ln | create another link (name) to a file |
chmod | set file permissions |
crypt | encode/decode a file with a private key |
gzip, gunzip | compress/decompress a file |
find | find files that match specific criteria |
Display contents of files
cat | copy files to display device |
more | show text file on display terminal with paging control |
head | show first few lines of a file(s) |
tail | show last few lines of a file; or reverse line order |
vi | full-featured screen editor for modifying text files |
pico | simple screen editor for modifying text files |
grep | display lines that match a pattern |
lpr | send file to printer |
pr | format file with page headers, multiple columns, etc. |
diff | compare two files and show differences |
cmp | compare two binary files and report if different |
comm | compare two files; show common or unique lines |
od | display binary files as eqivalent octal/hex codes |
strings | show printable text embedded in binary files |
file | examine file(s) and guess type: text, data, program, etc. |
wc | count characters, words, and lines in a file |
Directories
cd | change to new directory |
mkdir | create new directory |
rmdir | remove empty directory (you must remove files first) |
mv | change name of directory |
pwd | show current directory |
Disks
df | summarize free space on disk filesystems |
du | show disk space used by files or directories |
Special character handling for C-shell (See man csh)
* | match any characters in a file name |
~user | shorthand for home directory of user |
$name | substitute value of variable name |
\ | turn off special meaning of character that follows |
‘ | in pairs, quote string with special chars, except ! |
“ | in pairs, quote string with special chars, except !, $ |
` | in pairs, substitute output from enclosed command |
Controlling program execution for C-shell (See man csh)
& | run job in background |
^c | kill job in foreground |
^z | suspend job in foreground |
fg | restart suspended job in foreground |
bg | run suspended job in background |
; | delimit commands on same line |
() | group commands on same line |
! | re-run earlier commands from history list |
jobs | list current jobs |
ps | show process information |
kill | kill background job or previous process |
nice | run program at lower priority |
at | run program at a later time |
crontab | run program at specified intervals |
limit | see or set resource limits for programs |
alias | create alias name for program (normally used in .login file) |
sh, csh | execute command file |
Controlling program input/output for C-shell (See man csh)
| | pipe output to input |
> | redirect output to a storage file |
< | redirect input from a storage file |
>> | append redirected output to a storage file |
tee | copy input to both file and next program in pipe |
script | make file record of all terminal activity |
Email and communication
msgs | read system bulletin board messages |
mailx | send/read email; can be run by other programs to send exisiting files via email |
uuencode uudecode |
encode/decode a binary file for transmission via email |
rn | read USENET news groups |
Editors and formatting utilities
sed | programmable text editor for data streams |
vi | full-featured editor for character terminals |
emacs | GNU emacs editor for character terminals |
xemacs | GNU emacs editor for X Window terminals |
pico | very simple text editor |
fmt | fill and break lines to make all same length |
fold | break long lines to specified length |
X Window client programs (output to X terminal or server)
xterm | provide login shell window |
xauth | manipulate authorization files |
xload | show system load |
xman | full screen online manual viewer |
xemacs | GNU emacs editor |
gv | interface to contol gs to display PostScript or PDF files on screen |
xdvi | display DVI files on X Window (screen preview) |
gnuplot | interactive data plotting on screen |
TeX typesetting system
tex | process TeX files to DVI (device independent) output |
latex | process LaTeX files to DVI |
dvips | print DVI files on Postscript laser printer |
xdvi | display DVI files on X Window (screen preview) |
latex2html | translate LaTeX files to HTML (for web pages) |
Printing (BSD based)
lpr | send file to print queue |
lpq | examine status of files in print queue |
lprm | remove a file from print queue |
enscript | convert text files to PostScript format for printing |
Interpreted languages and data manipulation utilities
sed | programmable text editor for data streams |
awk | pattern scanning and processing language |
perl | Practical Extraction and Report Language |
sort | sort or merge lines in a file(s) by specified fields |
tr | translate characters |
cut | cut out columns from a file |
paste | paste columns into a file |
dd | copy data between devices; reblock; convert EBCDIC |
Graphics and mapping
gnuplot | interactive data plotting; outputs to PostScript or X Window |
gs | “ghostscript” converter displays PostScript files on X Window displays or other devices |
Networking/communications
ssh | remote login/command execution; encrypted |
scp | remote non-interactive file copy; encrypted |
sftp | remote interactive file copy; encrypted |
telnet | remote network login; plain text password – not recommended |
ftp | network file transfer program; plain text password – not recommended |
host | find IP address for given host name, or vice versa |
lynx | web browser for character based (text-only) terminals |
gzip, gunzip | compress/decompress a file |
tar | combine multiple files/dirs into single archive |
uuencode uudecode |
encode/decode a binary file for transmission via email |
Compilers, interpreters and programming tools
csh | command language interpreter (C-shell scripts) |
ksh | command language interpreter (Korn-shell scripts) |
sh | command language interpreter (Borne-shell scripts) |
f77 | Fortran 77 compiler |
f2c | convert fortran source code to C source code |
gcc | GNU C compiler |
g++ | GNU C++ compiler |
dbx | command-line symbolic debugger for compiled C or Fortran |
make | recompile programs from modified source |
cflow | generate C flow graph |
Programming libraries (see man library_name)
lapack | Fortran 77 routines for numerical linear algebra (supersedes LINPACK and EISPACK) |
X | routines to interface with X window system (no man page — get the X Toolkit book) |
dbm | database routines |
xdr | library routines for external data representation |
netcdf | routines for machine independent data representation |
Tape manipulation and archiving
mt | manipulate tape drive and position tape |
dd | unformatted tape read and write; file conversion |
tar | archive disk files on tape or disk |
ltf | read/write ANSI standard label tapes |