Tag: Linux
How to Set or Configure Proxy in Linux and Windows System? – scmGalaxy
Setting the proxy configuration in Linux and Windows
If you use a proxy server or firewall, you may need to set the http_proxy environment variable in order to access some url from command-line.
Windows Command line
set http_proxy=http://your_proxy:your_port
set http_proxy=http://username:password@your_proxy:your_port
set https_proxy=https://your_proxy:your_port
set https_proxy=https://username:password@your_proxy:your_port
Windows GUI
1. Open the Control Panel and click the System icon.The System Properties dialog is displayed.
2. On the Advanced tab, click on Environment Variables. The Environment Variables dialog is displayed.
3. Click New in the System variables panel. The New Sytem Variable dialog is displayed.
4. Add http_proxy with the appropriate proxy information
Windows Registry
IE can set username and password proxies, so maybe setting it there and import does work
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyEnable /t REG_DWORD /d 1
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyServer /t REG_SZ /d name:port
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyUser /t REG_SZ /d username
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyPass /t REG_SZ /d password
netsh winhttp import proxy source=ie
Command to enable proxy usage:
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyEnable /t REG_DWORD /d 1 /f
Command to disable proxy usage:
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyEnable /t REG_DWORD /d 0 /f
Command to change the proxy address:
reg add “HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings” /v ProxyServer /t REG_SZ /d proxyserveraddress:proxyport /f
Linux
export http_proxy=http://your_proxy:your_port
export http_proxy=http://username:password@your_proxy:your_port
export https_proxy=https://your_proxy:your_port
export https_proxy=https://username:password@your_proxy:your_port
export https_proxy=https://%username%:%password%@your_proxy:your_port
FAQ
1. How to escape if password has a @ character
Ans – try %40 instead of @
2. What is the file name where it stroed the proxy in Ubantu
Ans – /etc/environment
3. How to set proxy inforamtin in Apt?
Ans – Adding following line to /etc/apt/apt.conf has solved the problem:
Acquire::http::proxy “http://10.1.3.1:8080/”;
If file does not exist, create it. Do not confuse it with apt.conf.d directory.
4. How to set proxy inforamtin in linux Profile?
5. Why manual export failed to affect apt-get with the proxy info?
Ans – The reason your manual export failed to affect apt-get is because sudo ignores that environment variable by default (i.e. it doesn’t pass it on to the command). For one-off runs, you could do sudo env http_proxy=http://10.1.3.1:8080 apt-get update. Otherwise, you could configure sudo to allow http_proxy to fall through.
Java Installation Process in Linux – Complete guide
Download, Install and Configure JDK 8 & JRE 8
Platfrom – Debian & Ubuntu
#JRE8 - Package contains just the Java Runtime Environment 8 $ sudo apt-get install openjdk-8-jre #JKD8 - Package contains just the Java Developement Environment 8 $ sudo apt-get install openjdk-8-jdk
Platfrom – Fedora, Oracle Linux, Red Hat Enterprise Linux, etc
#JRE8 - Package contains just the Java Runtime Environment 8 $ su -c “yum install java-1.8.0-openjdk” #JKD8 - Package contains just the Java Developement Environment 8 $ su -c "yum install java-1.8.0-openjdk-devel" $ wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u151-b12/e758a0de34e24606bca991d704f6dcbf/jdk-8u151-linux-x64.rpm" $ wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.rpm curl -v -j -k -L -H "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.rpm > jdk-8u112-linux-x64.rpm
Platfrom – All platforms of Linux, Windows and Mac in Tar ball format
$ wget --no-check-certificate -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u151-b12/e758a0de34e24606bca991d704f6dcbf/jdk-8u151-linux-x64.tar.gz $ wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u151-b12/e758a0de34e24606bca991d704f6dcbf/jdk-8u151-linux-x64.tar.gz" $ wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.tar.gz
How to set JAVA in Linux System?
$ export JAVA_HOME=/opt/jdk1.8.0_144/ $ export PATH=/opt/jdk1.8.0_144/bin:$PATH;
Download, Install and Configure JDK 7 & JRE 7
Platfrom – Debian & Ubuntu
#JRE7 - Package contains just the Java Runtime Environment 7 $ sudo apt-get install openjdk-7-jre #JKD7 - Package contains just the Java Developement Environment 7 $ sudo apt-get install openjdk-7-jdk
Platfrom – Fedora, Oracle Linux, Red Hat Enterprise Linux, etc
$ su -c “yum install java-1.7.0-openjdk” $ su -c “yum install java-1.7.0-openjdk-devel”
Platfrom – All platforms of Linux, Windows and Mac in Tar ball format
wget –no-cookies –header “Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com” “http://download.oracle.com/otn-pub/java/jdk/7/jdk-7-linux-x64.tar.gz” wget –no-check-certificate –no-cookies –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz curl -v -j -k -L -H “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.rpm > jdk-7u79-linux-x64.rpm
JDK 6
Debian, Ubuntu, etc.
On the command line, type:
$ sudo apt-get install openjdk-6-jre
The openjdk-6-jre package contains just the Java Runtime Environment.
$ sudo apt-get install openjdk-6-jdk
If you want to develop Java programs then install the openjdk-6-jdk package.
Fedora, Oracle Linux, Red Hat Enterprise Linux, etc.
On the command line, type:
$ su -c “yum install java-1.6.0-openjdk”
The java-1.6.0-openjdk package contains just the Java Runtime Environment.
$ su -c “yum install java-1.6.0-openjdk-devel”
If you want to develop Java programs then install the java-1.6.0-openjdk-devel package.
Complete Linux & Shell Scripting Guide and Tutorial for Linux Admin and DevOps Engineer
Linux User Commands
Linux Admin Commands
- Linux Commands for Administrator
- Main Responsibilities of the Linux System Administrator
- 20 pmap Commands Examples in Linux / UNIX
- 20 mpstat Commands Examples in Linux / UNIX
- 20 iostat Commands Examples in Linux / UNIX
- 20 vmstat Commands Examples in Linux / UNIX
- 20 Xargs Commands Examples in Linux / UNIX
- 20 Mount and Unmount Filesystem / Partition commands in Linux / UNIX
- Steps to Perform SSH Login Without Password Using ssh-keygen and ssh-copy-id
- Linux Performance Monitoring using iostat, mpstat and vmstat
- Understanding Epoch & Unix Timestamp
- Common unix commands and utilities
- Shell Script Parameters
- Unix Sed Quick Reference
- Reading XML file using shell script
- Unix Command: Grep
- VI/VIM editor Commands
- Basic of Unix Presentation
Useful Tools in Linux
Linux Shell Scripting Collection and Interview Guide
- Interview Questions Sets : Shell Script Descriptive
- Interview Questions & Answer Sets : Shell Programming
- Must have collection of shell script for any SCM admin
- Frequently Used Shell Script
- Shell Script collection in scmGalaxy Forum
- Automate mandatory user-parameter in Shell Script using expect
- Various Commands to send an email in Linux
- df advance command usage
- FTP commands for Linux / Unix
- ShellScript to monitor available disk space on AIX
- Find Linux Command Collection
- Disable IPv6 and Enable IPv4 in Red Hat Linux
- Configuring NFS to access the files from remote Linux machine as a mount point
- Linux Shell Scripting Units and Exercise Unit – 1
- Linux Shell Scripting Units and Exercise Unit – 2
- Linux Shell Scripting Units and Exercise Unit – 3
- Linux Shell Scripting Units and Exercise Unit – 4
- Linux Shell Scripting Units and Exercise Unit – 5
Linux Troubleshooting Guide
Linux Quiz
Linux Exercise
Linux Bash Scripting Video Tutorial and CBT
How to Install Docker in Linux?
How to Install Docker in Linux?
Note – You should install Docker using root or with sudo access.
Install Docker on Ubantu
# apt-get update
# apt-get install -y docker.io
# service docker.io start
Install Docker on Redhar / Centos
# yum update
# yum install -y docker.io
# systemctl start docker.service
How to verify the version of docker?
# docker -v
# docker version
How to know docker running?
# service docker.io status (Ubantu)
# systemctl status docker.service (Redhat)
How to check details of Docker clients, deamon, containers, images, drivers, etc
# docker info
How to update Docker version?
==============================================
# wget -q0- https://get.docker.com/gpg | apt-key add –
# echo deb http://get.docker.com/ubantu docker main > /etc/apt/sources.list.d/docker.list
# apt-get update
# apt-get install lxc-docker
# docker version
Configuration post docker installation:
Adding Users to the Docker Group for non-root user
==============================================
# docker run -it ubuntu /bin/bash (as a non-root)
[ permission denied]
# cat /etc/group
# sudo gpasswd -a username docker
# cat /etc/group
# docker run -it ubuntu /bin/bash (as a non-root)
# logout
# login username
Location of Docker images in all Operating Systems (Linux, Windows, Redhat, Mac OS X)
Location of Dockers images in all Operating Systems
The location of the images vary depending on the driver Docker is using for storage. Actually, Docker images are stored in two files as shown by following command.
# docker info
aufs – Most linux version
/var/lib/docker/aufs/diff/<id> has the file contents of the images.
/var/lib/docker/repositories-aufs is a JSON file containing local image information. This can be viewed with the command docker images.
btrfs
TBD
devicemapper (Redhat)
/var/lib/docker/devicemapper/devicemapper/data – stores the images
/var/lib/docker/devicemapper/devicemapper/metadata – the metadata
vfs
TBD
Mac OS X
~/VirtualBox VMs/boot2docker-vm
Mac OS X using boot2docker
/Users/rajesh.kumar/.docker/machine/machines/default
File Name – disk.vmdk
Windows:
%USERPROFILE%/VirtualBox VMs/boot2docker-vm
You can manually set the storage driver with the -s or –storage-driver= option to the Docker daemon.
/var/lib/docker/{driver-name} will contain the driver specific storage for contents of the images.
/var/lib/docker/graph/<id> now only contains metadata about the image, in the json and layersize files.
Reference
20 pmap Commands Examples in Linux / UNIX | pmap Commands Tutorial
You can find the memory used by a program (process) by looking into /proc directory or using standard command such as ps or top.
However, you must calculate all memory usage by hand i.e. add Shared Memory + mapped file + total virtual memory size of the process + Resident Set Size + non-swapped physical memory used by process. So how do you find the total memory used by a process or program under Linux? Use a tool called pmap. It reports the memory map of a process or processes.
To display process mappings, type
$ pmap pid
$ pmap 3724
If you want to see the exented format
The -x option can be used to provide information about the memory allocation and mapping types per mapping. The amount of resident, non-shared anonymous, and locked memory is shown for each mapping:
The -x option can be used to provide information about the memory allocation and mapping types per mapping. The amount of resident, non-shared anonymous, and locked memory is shown for each mapping:
$ pmap -x 3526
If you want to see the device format
$ pmap -d 3526
If you do not display some header/footer lines.
$ pmap -q 3526
GENERAL OPTIONS
-x extended Show the extended format.
-d device Show the device format.
-q quiet Do not display some header/footer lines.
-V show version Displays version of program.
Reference
20 mpstat Commands Examples in Linux / UNIX | mpstat Commands Tutorial
If you are using SMP (Multiple CPU) system, use mpstat command to display the utilization of each CPU individually.
mpstat Report processors related statistics which includes Collecting and displays performance statistics for all logical processors in the system.. This display CPU statistics of individual CPU (or) Core.
How to install mpstat?
1. Ensure you are logged in as ‘root’
‘mpstat’ and ‘vmstat’ are apart of the ‘sysstat’ package and should be installed by default
2. Verify installation :
# rpm -qa | grep -i sysstat
3. Installation, if needed :
# rpm -ivh <name>
4. If you have connectivity to RHN or a Satellite Server:
# yum install sysstat
5. If you are using Ubantu:
# apt-get install sysstat
Usage:
1. Using mpstat command without any option, will display the Global Average Activities by All CPUs.
$ mpstat
2. Using mpstat with option ‘-P’ (Indicate Processor Number) and ‘ALL’, will display statistics about all CPUs one by one starting from 0. 0 will the first one.
$ mpstat -P ALL
3. To display the statistics for N number of iterations after n seconds interval with average of each cpu use the following command.
$ mpstat -P ALL 2 5
4. The option ‘I’ will print total number of interrupt statistics about per processor.
$ mpstat -I
5. Get all the above information in one command i.e. equivalent to “-u -I ALL -p ALL”.
$ mpstat -A
Reference
20 iostat Commands Examples in Linux / UNIX | iostat Commands Tutorial
iostat command is a command that used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates. The iostat create reports that can be used to change system configuration to better balance the input/output between physical disks.
You can use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It can be use to find out your system’s average CPU utilization since the last reboot.
Reports Central Processing Unit (CPU) statistics, asynchronous input/output (AIO) and input/output statistics for the entire system, adapters, TTY devices, disks CD-ROMs, tapes and file systems.
Note:
iostat reports CPU, disk I/O, and NFS statistics.
vmstat reports virtual memory statistics.
mpstat reports processors statictics.
How to install iostat?
1. Ensure you are logged in as ‘root’
‘iostat’ and ‘vmstat’ are apart of the ‘sysstat’ package and should be installed by default
2. Verify installation :
# rpm -qa | grep -i sysstat
3. Installation, if needed :
# rpm -ivh <name>
4. If you have connectivity to RHN or a Satellite Server:
# yum install sysstat
5. If you are using Ubantu:
# apt-get install sysstat
1. iostat – Basic example, Iostat without any argument displays information about the CPU usage, and I/O statistics about all the partitions on the system as shown below.
$ iostat
2. iostat – Display only cpu statistics. iostat option -c, displays only the CPU usage statistics as shown below.
$ iostat -c
3. iostat – Display only disk I/O statistics. iostat option -d, displays only the disk I/O statistics as shown below.
$ iostat -d
4. iostat – Display only network statistics. iostat option -n, displays only the device and NFS statistics as shown below.
$ iostat -n
5. iostat – Display I/O data in MB/second. By default iostat, displays the device I/O statistics in Blocks. To change it to MB, use -m as shown below.
$ iostat -m
6. iostat – Display I/O statistics only for a device. By default iostat displays I/O data for all the disks available in the system. To view statistics for a specific device (For example, /dev/sda), use the option -p as shown below.
$ iostat -p sda
7. iostat – Display timestamp information, By default iostat displays only the current date. To display the current time, use the option -t as shown below.
$ $ iostat -t
8. iostat – Display Extended status, Use option -x, which will displays extended disk I/O statistics information as shown below.
$ iostat -x
9. To display extended information for a specific partition (For example, /dev/sda1), do the following.
$ iostat -x sda1
10. iostat – Execute Every x seconds (for y number of times). To execute iostat every 2 seconds (until you press Ctl-C), do the following.
$ iostat 2
11.To execute every 2 seconds for a total of 3 times, do the following.
$ iostat 2 3
12. iostat – Display LVM statistic (and version)
$ iostat -N
13. To display the version of iostat, use -V. This will really display the version information of sysstat, as iostat is part of sysstat package.
$ iostat -V
Reference
iostat use these files to create reports…
/proc/stat which contains system statistics
/proc/partitions which contains disk statistics (for pre 2.5 kernel that have been patched)
/proc/diskstats contains disks statistics (for post 2.5 kernel)
/sys which contains statistics for block devices (post 2.5 kernel)
20 vmstat Commands Examples in Linux / UNIX | vmstat Commands Tutorials
vmstat – Report virtual memory statistics. vmstat is a tool that collects and reports data about your system’s memory, swap, and processor resource utilization in real time. It can be used to determine the root cause of performance and issues related to memory use.
vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity. The first report produced gives averages since the last reboot. Additional reports give information on a sampling period of length delay. The process and memory reports are instantaneous in either case.
Note:
iostat reports CPU, disk I/O, and NFS statistics.
vmstat reports virtual memory statistics.
mpstat reports processors statictics.
How to install vmstat?
1. Ensure you are logged in as ‘root’
‘iostat’ and ‘vmstat’ are apart of the ‘sysstat’ package and should be installed by default
2. Verify installation :
# rpm -qa | grep -i sysstat
3. Installation, if needed :
# rpm -ivh <name>
4. If you have connectivity to RHN or a Satellite Server:
# yum install sysstat
5. If you are using Ubantu:
# apt-get install sysstat
Example
1. The significant of the columns are explained in man page of vmstat in details. Most important fields are free under memory and si, so under swap column.
[root@tecmint ~]# vmstat -a
procs ———–memory———- —swap– —–io—- –system– —–cpu—–
r b swpd free inact active si so bi bo in cs us sy id wa st
1 0 0 810420 97380 70628 0 0 115 4 89 79 1 6 90 3 0
Free – Amount of free/idle memory spaces.
si – Swaped in every second from disk in Kilo Bytes.
so – Swaped out every second to disk in Kilo Bytes.
2. Execute vmstat ‘X’ seconds and (‘N’number of times). With this command, vmstat execute every two seconds and stop automatically after executing six intervals.
# vmstat 2 6
3. Vmstat with timestamps. vmstat command with -t parameter shows timestamps with every line printed as shown below.
# vmstat -t 1 5
4. Statistics of Various Counter, vmstat command and -s switch displays summary of various event counters and memory statistics.
$ vmstat -s
5. Disks Statistics, vmstat with -d option display all disks statistics.
$ vmstat -d
6. Display Statistics in Megabytes, The vmstat displays in Megabytes with parameters -S and M(Uppercase & megabytes). By default vmstat displays statistics in kilobytes.
# vmstat -S M 1 5
7. vmstat – Display number of forks since last boot, This displays all the fork system calls made by the system since the last boot. This displays all fork, vfork, and clone system call counts.
# vmstat -f
8. vmstat – Display slab info, Use option -m, to display the slab info as shown below.
# vmstat -m
Reference
The output of vmstat is displayed in a number of columns. The following sections provide brief overviews of the data reported in each column.
Procs
The procs data reports the number of processing jobs waiting to run and allows you to determine if there are processes “blocking” your system from running smoothly.
The r column displays the total number of processes waiting for access to the processor. The b column displays the total number of processes in a “sleep” state.
These values are often 0.
r: The number of processes waiting for run time.
b: The number of processes in uninterruptible sleep.
Memory
The information displayed in the memory section provides the same data about memory usage as the command free -m.
The swapd or “swapped” column reports how much memory has been swapped out to a swap file or disk. The free column reports the amount of unallocated memory. The buff or “buffers” column reports the amount of allocated memory in use. The cache column reports the amount of allocated memory that could be swapped to disk or unallocated if the resources are needed for another task.
swpd: the amount of virtual memory used.
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)
Swap
The swap section reports the rate that memory is sent to or retrieved from the swap system. By reporting “swapping” separately from total disk activity, vmstat allows you to determine how much disk activity is related to the swap system.
The si column reports the amount of memory that is moved from swap to “real” memory per second. The so column reports the amount of memory that is moved to swap from “real” memory per second.
si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).
I/O
The io section reports the amount of input and output activity per second in terms of blocks read and blocks written.
The bi column reports the number of blocks received, or “blocks in”, from a disk per second. The bo column reports the number of blocks sent, or “blocks out”, to a disk per second.
bi: Blocks received from a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).
System
The system section reports data that reflects the number of system operations per second.
The in column reports the number of system interrupts per second, including interrupts from system clock. The cs column reports the number of context switches that the system makes in order to process all tasks.
in: The number of interrupts per second, including the clock.
cs: The number of context switches per second.
CPU
The cpu section reports on the use of the system’s CPU resources. The columns in this section always add to 100 and reflect “percentage of available time”.
The us column reports the amount of time that the processor spends on userland tasks, or all non-kernel processes. The sy column reports the amount of time that the processor spends on kernel related tasks. The id column reports the amount of time that the processor spends idle. The wa column reports the amount of time that the processor spends waiting for IO operations to complete before being able to continue processing tasks.
These are percentages of total CPU time.
us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.
Field Description For Disk Mode
Reads
total: Total reads completed successfully
merged: grouped reads (resulting in one I/O)
sectors: Sectors read successfully
ms: milliseconds spent reading
Writes
total: Total writes completed successfully
merged: grouped writes (resulting in one I/O)
sectors: Sectors written successfully
ms: milliseconds spent writing
IO
cur: I/O in progress
s: seconds spent for I/O
Field Description For Disk Partition Mode
reads: Total number of reads issued to this partition
read sectors: Total read sectors for partition
writes : Total number of writes issued to this partition
requested writes: Total number of write requests made for partition
Field Description For Slab Mode
cache: Cache name
num: Number of currently active objects
total: Total number of available objects
size: Size of each object
pages: Number of pages with at least one active object
totpages: Total number of allocated pages
pslab: Number of pages per slab
Files
/proc/meminfo
/proc/stat
/proc/*/stat