Tag: server

gsh – Run linux commands on many other linux server at once

by scmgalaxy KUnix / LinuxLeave a Comment on gsh – Run linux commands on many other linux server at once

gsh-run-linux-commands

gsh – Run linux commands on many other linux server at once

Group Shell (also called gsh) is a remote shell multiplexor. It lets you control many remote shells at once in a single shell. Unlike other commands dispatchers, it is interactive, so shells spawned on the remote hosts are persistent.

GSH is a pluggable version of DSH (Distributed Shell) written in Python. Both a module and a command-line tool for running a shell command over multiple machines are included. GSH can be extended by adding new host loaders as well as hooking into various stages of the runtime.

It requires only a SSH server on the remote hosts, or some other way to open a remote shell.
gsh allows you to run commands on multiple hosts by adding tags to the gsh command.

e.g > gsh tag “remote command”

Important things to remember:
/etc/ghosts contains a list of all the servers and tags. gsh is a lot more fun once you’ve set up ssh keys to your servers

gsh [OPTIONS] SYSTEMS CMD…
SYSTEMS is a combination of ghost macros. See ghosts(1).

CMD is the command to run
-h, –help Display full help
-d, –debug Turn on exeuction debugging reports
-h, –no-host-prefix Does not prefix output lines with the host name
-s, –show-commands Displays the command before the output report
-n, –open-stdin Leaves stdin open when running (scary!)
-l, –user USER SSH’s to the host as user USER
-r, –run-locally Run commands locally (replaces $host with host)
-o, –self-remote Run locally instead of over SSH for local host
-V, –version Report the version and exit

You set up a /etc/ghosts file containing your servers in groups such as web, db, RHEL4, x86_64, or whatever (man ghosts) then you use that group when you call gsh.

How to run the gsh?
> gsh linux “cat /etc/redhat-release; uname -r”

You can also combine or split ghost groups, using cpanel+vps or web-RHEL4, for example.

Here’s an example /etc/ghosts file:
# Machines
 #
 # hostname OS-Version Hardware OS cp security
 linuxbrigade.com debian6 baremetal linux plesk iptables
 linuxbrigade.com centos5 vps linux cpanel csfcluster
 linuxbrigade.com debian7 baremetal linux plesk iptables
 linuxbrigade.com centos6 vps linux cpanel csfcluster
 linuxbrigade.com centos6 vps linux cpanel csfcluster
 linuxbrigade.com centos6 vps linux nocp denyhosts
 linuxbrigade.com debian6 baremetal linux plesk iptables
 linuxbrigade.com centos6 baremetal linux cpanel csf
 linuxbrigade.com centos5 vps linux cpanel csf

Reference url

http://www.linuxbrigade.com/run-the-same-command-on-many-linux-servers-at-once/
http://linux.die.net/man/1/gsh
https://github.com/gmjosack/gsh
http://outflux.net/unix/software/gsh/
http://outflux.net/unix/software/gsh/gsh.html

Tagged : / / / / / / / / / /

If a linux-build-server suddenly starts getting slow, what would you check?

by scmgalaxy KUnix - LinuxLeave a Comment on If a linux-build-server suddenly starts getting slow, what would you check?

linux-build-server

If a linux-build-server suddenly starts getting slow, I would divide my approach / troubleshooting into 3 section as follows;

1. System Level troubleshooting

a. RAM related issues

b. Disk Space related Issues

c. Disk I/O read write issues

d. Network Hardware issues

e. Mount issues

f. Too Many process running in the machine

2. Application Level troubleshooting

a. Application is not behaving properly. Hit to Application log file OR application server log file OR web server Log file and try to understand the issues.

b. zombie process issues – Find out if any as such process which is causing the system performance issues.

c. Application Log – depends on the application installed, this can be referred and make use of the experience with the project and troubleshoot.

d. Web Server Log – we can check http, tomcat log as well.

e. Application Server Log – We can see jboss, weblogic logs to see if the application server response/receive time is the issues for slowness.

f. Memory Leak of any application – This is one of well known issues in lunux based server due to bad application coding. Many times this can be resolved either by fixing the code or rebooting. But many other solutions are there to apply.

3. Dependent Services troubleshooting

a. SMTP Response time – SMTP server is not responding faster which is causing delay in response and queue up many processes.

b. Network issues – There are many System performance issues is dependent on network or service which is depends on the network.

c. Firewall related issues

d. Antivirus related issues

 Some of the useful commands for troubleshooting are..

1. df –k

2. du –sh

3. top

4. uptime

5. ps –eaf | grep

6. vmstat

7. ping

8. tail –f <logfile>

9. iostat

10.free

11.kill -9

12.mount

13.sar

14.ifconfig eth0 | enable | disable

15.traceroute

16.netstat -r

17.nslookup

18.route

   There are various options available for each commands and depends on the need during the troubleshooting, right commands can be applied. To find arguments and their explanation, I usually refer google.com rather than man page.

Tagged : / / / / / / / / / / / / / / / / /

Ways to Perforce server Disk Space Cleanup and Repos Size Management

by scmgalaxy KPerforceLeave a Comment on Ways to Perforce server Disk Space Cleanup and Repos Size Management

perforce-server-disk-space-cleanup-and-repos-size-management

DRAFT VERSION

  1. Cleaning up Old Checkpoints
  2. Playing with Symlink(Softlink) and Redirecting the ROOT folder to drive where we have enough place.
  3. Deleting db.have and recreate it manually
  4. Display disk space information on the server using “p4 diskspace”
  5. Display size information for files in the depot using “p4 sizes”
Tagged : / / / / / / / / / / / / / / / / /

Buildforge common Issues and Troubleshooting | Buildforge Troubleshooting Guide

by scmgalaxy KBuildForgeLeave a Comment on Buildforge common Issues and Troubleshooting | Buildforge Troubleshooting Guide

buildforge-troubleshooting

Know about the BuildForge Server before Troubleshooting.

1. What is the Full version of Build Forge being used
(for example Build Forge  7.1.2.2-1-0111).This can be obtained under the console interface by hovering the mouse over the Build Forge icon in the upper left hand corner.

2. What is Full version of operating system for both the Management Console and the agent host (used for the selector)

for example Windows XP Professional SP3 32 bit)
> AIX = oslevel
> HP-UX = uname -r
> Solaris = uname -r
> Red Hat = cat /etc/redhat-release
> SUSE = cat /etc/SuSE-release
> Windows = winver (in a command (cmd) window)

3. What is the Full version of the database being used.

4. In What situtation and how, When did the behavior first start to occur?

5. Need to Refer Tomcat (Catalina) logs on the Build forge Server. 
UNIX/Linux ($BF_HOME/server/tomcat/logs/log_from_date_of_error.log) We will only require the Tomcat Catalina log for today’s date or the date you first encountered the error, i.e. catalina.2012-01-17.log

6. Need to Refer Build Forge engine log
UNIX/Linux ($BF_HOME/log)

7. Need to review database log
UNIX/Linux ($BF_HOME/Platform/db.log)

Issues 1 – API: Access denied.
Solution – Click Here

Issues 2: Build forge Consol gets slow after login

Issues 3: API: Access denied while accessing Environments Menu.

Issues 4: Error message in catalina.DATE.log”A valid Engine Unique Identifier must be specified”

Issues 5: Once copying one Environment varibale and duplicating multiple times. Error Occured after copying 7th times same variables. Post login, once You clicks on “Environments” left navigation, Everyone gets Error called “API: Access denied” on page.

 

Tagged : / / / / / / / / / / /

Performance Optimization of Build Server | Performance Optimization Guide

by scmgalaxy KBuild ManagementLeave a Comment on Performance Optimization of Build Server | Performance Optimization Guide

build-server

Performance Optimization Checklist of Build Servers

Build Infrastructure Level

1.  Do you really need to build all source code or only the part of code which has changes?

2. Project should be divided into multiple modules/component which can be build independently and should be integrated when it needed.

3. Understand technology and make use of performance related features which will help you. e.g make, visual studio and maven has some feature which help us to compile only code which was changed also, some performance tuning features should be explored.

4. Compile dependent project only there are changes. no need to compile all the time.

5. Distribute application in non-compiled and to be compiled files. e.g src, docs and copy only src to comile. docs files cane be used while creating installer.

6. Plan for disk usage growth upfront.

Application Server level

  1. Apache Server optimization and proper log setting
  2. Disabling unused modules and component which will help performance.
  3. Follow best practices of Module Configuration, Apache Server Configuration etc

 System level

  1. Check if any background task/services is running which is consuming cpu/memory and can be  stopped and not relevant.
  2. Keep system uptodate
  3. Avoid using build server for any other task – If you must edit components or install software, build or edit the components on another machine and copy them over.
  4. Use high speen drive for i/o operations

Build Server application level

http://www.cloudbees.com/sites/default/files/whitepapers/7WaysToOptimizeJenkins.pdf

  1. Keep log files maintenance and truncate regularly. have limited build records.
  2. Move build to document server. not to keep in build server
  3. Take advantage of distribute build server using agents/slaves/nodes etc
Tagged : / / / / / / / / / / / / / / /

Hardware Configuration – What are the Hardware Requirements for Build Server?

by scmgalaxy KBuild Management & ToolsLeave a Comment on Hardware Configuration – What are the Hardware Requirements for Build Server?

build-server-hardware-configuration-requirements

Hardware Configuration – Hardware Requirements for Build Server

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

At a minimum, the build lab should have four machines:

Server that contains the Source Code Control program— This is your product. Do you really want this server residing someplace where you have little control over this box?

Debug build machine for the mainline builds— If you don’t separate your debug and release machines, you will accidentally ship debug binaries, which is not a good thing.

Release build machine for the mainline builds— This is a “golden goose” that turns out the “gold eggs” of your company or group. Treasure this machine like a princess, and guard it like all the king’s fortunes.

Internal release share server— This is one more piece of hardware that stores the “bread and butter” of the group or company. Don’t give up control of this hardware to anyone unless your IT department reports through your development group.

Hardware Requirements
Each machine in the preceding list should meet the following requirements:

Number of processors— This depends on the build tool you use. One is usually sufficient, because few build tools really take advantage of multiple processors.

Processor speed— The lab budget dictates this, but the faster the processor, the better it is.

Amount of installed RAM— Max out the machine. RAM is relatively cheap these days, especially when you consider the performance increase you get. Increasing the RAM is usually the first upgrade done when trying to improve the performance of any computer.

Number of hard drives— A minimum of two drives (or partitions) is preferred:

Drive 1 (C:) is for the operating system and installed applications.

Drive 2 (D:) is for building binaries, release shares, or the source database; the minimum space required is roughly ten times the space needed to build your application.

The split partitions are good because if you ever need to format or blow away a drive due to corruption, only part of the project will be affected. The recovery is much faster and easier.

Hard drive type— This is most likely SCSI, but it could be IDE.

Number of power supplies— If you purchase server class hardware (pizza boxes) that belong in racks, you need to consider how many power supplies to order.

Motherboard BIOS version— This does make a difference. Make sure you note what is being used and standardize on it.

Tagged : / / / / / / / / / / / / / /

Different Perforce Tools and Their Short Summary | Know About Different Perforce tools

by scmgalaxy KConcept & Definition / PerforceLeave a Comment on Different Perforce Tools and Their Short Summary | Know About Different Perforce tools

different-perforce-tools

Different Perforce Tools and Their Short Summary

Clients
P4V: Visual Client – (Included in the P4V Installer)
Provides access to versioned files through a graphical interface and also includes tools for merging and visualizing code evolution.
P4Merge: Visual Merge Tool – (Included in the P4V Installer)
Provides graphical three-way merging and side-by-side file comparisons
P4: Command-Line Client – (Included in the Perforce Server Windows Installer)
(Included in the Perforce Server Windows Installer)
P4Web: Web Client – (Included in the P4Web Installer)
Provides convenient access to versioned files through popular web browsers

Server
P4D: Server – (Included in the Perforce Server Windows Installer)
Stores and manages access to versioned files, tracks user operations and records all activity in a centralized database.
P4P: Proxy Server – (Included in the Perforce Server Windows Installer)
A self-maintaining proxy server that caches versioned files remotely on distributed networks.

Plug-ins & Integrations
P4WSAD: Plug-in for Eclipse and WebSphere Studio
Access Perforce from within the Eclipse IDE and the Rational/WebSphere Studio WorkBench family of products
P4SCC: SCC Plug-in – (Included in the P4V Installer)
Enables you to perform Perforce operations from within IDEs that support the Microsoft SCC API including Visual Studio.
P4EXP: Plug-in for Windows Explorer – (Included in the P4V Installer)
Allows Windows users direct access to Perforce.
P4DTG: Defect Tracking Gateway – (Included in the P4DTG Installer)
Allows information to be shared between Perforce’s basic defect tracking system and external defect tracking systems.
P4GT: Plug-in for Graphical Tools
Provides seamless access to version control for files from within Adobe Photoshop, SoftImage XSI, Autodesk’s 3ds max, and Maya
P4OFC: Plug-in for Microsoft Office
Allows documents to be easily stored and managed in Perforce directly from Microsoft Word, Excel, PowerPoint and Project.

Tools & Utilities
P4Report: Reporting System
Supports leading tools such as Crystal Reports, Microsoft Access, and Microsoft Excel, or any reporting
tool that interfaces with an ODBC data source.
P4Thumb: Thumbnail Generator
Creates thumbnails of graphics files managed by Perforce and stores the thumbnails in the server for presentation in P4V.
P4FTP: FTP Plug-in
Allows FTP clients like Dreamweaver, Netscape, and Internet Explorer to access files in Perforce depots.’
Links to Download: http://www.perforce.com/perforce/downloads/platform.html

Good Video Tutorial Links
http://www.perforce.com/perforce/media_library/tutorials.html
http://www.perforce.com/perforce/demo/testdrive.html

Tagged : / / / / / / / / / / / / / / / / / /

Perforce Network Troubleshooting Guide | How to Resolve Perforce Network Issues?

by scmgalaxy KPerforceLeave a Comment on Perforce Network Troubleshooting Guide | How to Resolve Perforce Network Issues?

perforce-network-troubleshooting-guide

1. netstat -a
Check to make sure that the server is running. netstat -a gives a list of all processes listening on network ports. Look for lines that contain “LISTEN” and “1666” (or whatever port you have Perforce running on.)  If you do not see such a line, the server is not running.

2. p4 -p 127.0.0.1:1666 info
Verify that the server accepts local connections using the localhost address from the server machine.
If you cannot connect check and make sure P4PORT is set to “1666” for the server. This ensures the server is listening on all interfaces.  Setting P4PORT to ‘localhost:1666’ will set it to only allow connections from the local machine.

Note:
Make sure that it is set properly with ‘p4 set -S Perforce P4PORT’ and if it’s not, set it:
p4 set -S Perforce P4PORT=1666

On Linux/Mac/Unix, instead of using ‘p4 set’, you can either set the environment variable $P4PORT or use the ‘-p’ flag to p4d:
p4d -r $P4ROOT -p 1666 [other flags]

3. ping server
Verify network connectivity by pinging the server from the client. If you cannot ping the server, then either ICMP is being blocked or there is a network issue outside of Perforce.

4. telnet <server> <port>
Verify that the server port is reachable by “telnet <server> <port>”. This can give you a descriptive error or confirm a connection. Note that on Windows servers the telnet utility might not be available by default; if it is not available, you can install it from your Programs and Features control panel item. In the sidebar there is a Turn Windows Features On or Off, select it and then check Telnet Client in the list. Click OK.

5. Check port filters/firewalls settings
If you still cannot connect, then verify the TCP/IP properties settings for any port filters/firewalls. If there is a firewall, make sure that incoming connections to port 1666 are permitted, and that all existing outbound connections are permitted (the latter is usually standard). On Windows machines, go to Control Panel -> Windows Firewall, click on Advanced Settings, and then click on Inbound Rules on the sidebar. Make sure p4d is enabled.

6. DNS is resolving the IP address correctly
Check to make sure that DNS is resolving the IP address correctly. Pinging by DNS name is quick verification. “nslookup <hostname>” or “dig <hostname>” can also work.

7. using the IP address
Run the Perforce command using the IP address instead of DNS name. If IP address works, suspect DNS resolution problems or an incorrectly spelled hostname. If IP address does not work, suspect a problem with either host table entries or routing or other network problems. The commands “route” and “arp -a” can be helpful in this regard.

8. If problem is intermittent, suspect hardware, interface configuration, or congestion problems. Tools such as fping and wireshark are useful for uncovering these sorts of errors.

9. tracert <perforce_server
Check for inordinate delays with the traceroute command “tracert <perforce_server>”. On linux, it is called “traceroute”.

Reference…
http://kb.perforce.com/article/905

Tagged : / / / / / / / / / / / / / / / / /

Apache web server Installation Guide, know how to Install Apache web server on Windows 7?

by scmgalaxy KApp / Web ServersLeave a Comment on Apache web server Installation Guide, know how to Install Apache web server on Windows 7?

apache-web-server-on-windows-7-installation-guide

After a quick detour to install Notepad++, I’m on my way to installing Magento. Before I can get into the meat of it, though, I do require a few prerequisites.

From the installation instructions, installing Magento requires:

  • Apache 1.3.x or Apache 2
  • PHP 5.2.0 and above with Safe mode off
  • MySQL 4.1.20 and above

After a brief search, I happened upon a very handy site (Yes, my current dev machine runs Windows 7). No sense reinventing the wheel, so these instructions should do nicely.

I’ve chosen the latest stable release, 2.2.17, MD5-verified and ready to install.

Always one to put the right foot forward… The first thing I’ve done is to immediately download the wrong version (src) instead of the neatly prepped installer (msi) as per the instructions. Round 1 goes to my eagerness, but after a neat revisit to the Apache download page, I’m off in the right direction.

After kicking off the Apache installer, my first instinct was to twitch at the frozen installer progress (Did the instructions forget about that ghastly Windows 7 User Account Control?), but after only a brief pause, the UAC prompt has displayed and the Apache web server installation is back on track.

None the worse for wear, time for the next step.

PHP

I’ve pretty much kaboshed most of the PHP installation instructions, mostly due to its avoidance of the handy PHP for Windows installer. Pointing the installer to the Apache web server’s cgi-bin folder seems like a good idea.

A configuration change…

; Directory in which the loadable extensions (modules) reside.
; http://php.net/extension-dir
; extension_dir = “./”
; On windows:
extension_dir = “ext” ; Uncommented (removed the semicolon at the beginning of) this line

… and I’m back to the instructions. After an update to my environment variables, looks like it’s time for a reboot… and sleep.

I’ll be leaving the MySQL install (and testing Apache web server and PHP) ’til tomorrow.

Tagged : / / / / / / / / / / / / / / / / /

Apache Tomcat – Tomcat 7 Finalized – Tomcat 7 Introduction

by scmgalaxy KApp / Web ServersLeave a Comment on Apache Tomcat – Tomcat 7 Finalized – Tomcat 7 Introduction

tomcat-7

The volunteer developers behind Apache Tomcat have released version 7.0.6 of the open-source Java servlet container.

“This is the first stable release of the Tomcat 7 branch,” developer Mark Thomas wrote in an e-mail announcing the release on various Tomcat developer mailing lists.

While not a full application server, Tomcat implements the functionality described in the Java Enterprise Edition Web profile specifications. Most notably, it supports version 3.0 of the Servlet API (application programming interface) and version 2.2 of JavaServer Pages, both part of the recently ratified JEE 6. A servlet container manages Java-based applications that can be accessed from a Web browser.

The use of Servlet 3.0 will bring a lot of new capabilities to Tomcat, noted Tomcat contributor Tim Funk, in a presentation at the ApacheCon conference in Atlanta in November.

One big area of improvement is in configuration management for Web applications. Previous versions required all Web app configuration changes to be entered in a central file called web.xml, a process that led to unwieldy web.xml files as well as security risks.

The new approach, called annotations, breaks configuration instructions into multiple files. Separate files are kept for security, for configuring new connections, for initial parameters and for handling third-party libraries.

Other benefits of Servlet 3.0 include advanced cookie management and asynchronous thread support, which should increase the number of connections Tomcat can handle.

In preparation for the release, various beta versions of Tomcat 7 have been distributed since April 2010. Version 7.0.6 fixes some longtime memory leak issues and includes a number of other bug fixes as well.

In addition to making the source code and various binaries available, the development team has packaged a version of the software that can be embedded in other applications.

Tomcat is one of almost 150 open-source projects maintained by the Apache Software Foundation.

Tagged : / / / / / / / / / / / / / / / / /