Useful Tools for Chef
Chef Code Correctness (Chef Code Analysis)
Chef Unit Testing
Chef Integration Testing
Some other tools which can be used along with Chef
Some other chef & Knife tools
Chef Tools for Windows
What is Role?
A role is a way to define certain patterns and processes that exist across nodes in an organization as belonging to a single job function. Each role consists of zero (or more) attributes and a run-list. Each node can have zero (or more) roles assigned to it. When a role is run against a node, the configuration details of that node are compared against the attributes of the role, and then the contents of that role’s run-list are applied to the node’s configuration details. When a chef-client runs, it merges its own attributes and run-lists with those contained within each assigned role.
How to use Roles in Chef?
How to create Role?
Method 1: In Chef Server directly
> knife role create client1&
Add the run list e.g. “recipe[nginx]” under “run_list”
Save & exit
The role will be created in Chef Server.
Example
{ "name": "client1", "description": "", "json_class": "Chef::Role", "default_attributes": { }, "override_attributes": { }, "chef_type": "role", "run_list": [ "recipe[nginx]", "recipe[phpapp::web]" ], "env_run_lists": { } } Let’s download the role from the Chef server so we have it locally in a Chef repository.
> knife role show client1 -d -Fjson > roles/client1.json
Now, Lets bootstrap the node using knife with roles
> knife bootstrap --run-list "role[client1]" --sudo hostname
How to edit the roles in chef Server?
> knife role edit client1
Method 2: In local repo under chef-repo folder
> vi webserver.rbexample –
name "web_server" description "Role for web servers" run_list( "role[base]", "recipe{web_server]" ) & Then upload to chef server using following commands.
> knife role from file path/to/role/file
How Assigning Roles to Nodes?
> knife node list> knife node edit node_nameThis will bring up the node’s definition file, which will allow us to add a role to its run_list:
{ "name": "client1", "chef_environment": "_default", "normal": { "tags": [ ] }, "run_list": [ "recipe[nginx]" ] } For instance, we can replace our recipe with our role in this file:
{ "name": "client1", "chef_environment": "_default", "normal": { "tags": [ ] }, "run_list": [ "role[web_server]" ] } How to bootstrap node using role?
> knife bootstrap {{address}} --ssh-user {{user}} --ssh-password '{{password}}' --sudo --use-sudo-password --node-name node1 --run-list 'role[production]'> knife bootstrap --run-list "role[phpapp-web]" --sudo hostnameHow to run roles against nodes?
You can run chef-client on multiple nodes via knife ssh command like, To query for all nodes that have the webserver role and then use SSH to run the command sudo chef-client, enter:
> knife ssh "role:webserver" "sudo chef-client"To find the uptime of all of web servers running Ubuntu on the Amazon EC2 platform, enter:
> knife ssh "role:web" "uptime" -x ubuntu -a ec2.public_hostnameReference
http://docs.chef.io/roles.html
https://docs.chef.io/knife_ssh.html
https://docs.chef.io/knife_role.html
This tutorial is to setup the SVNServer from base using Apache2 in Ubantu only.
> sudo apt-get update
> sudo apt-get install subversion apache2 libapache2-svn apache2-utils
> sudo addgroup subversion
> sudo mkdir /home/svn
> sudo chown -R www-data:subversion /home/svn
> sudo chmod -R g+ws /home/svn
> sudo svnadmin create /home/svn/myWebsite
sudo vim /etc/apache2/mods-available/dav_svn.conf
& add the following entry at the end of file.
<Location /svn/myWebsite> DAV svn SVNPath /home/svn/myWebsite AuthType Basic AuthName "myWebsite subversion Repository" AuthUserFile /etc/subversion/passwd <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user </LimitExcept> </Location>
sudo htpasswd -c /etc/subversion/passwd rajesh
sudo chown -R www-data:subversion /home/svn/myWebsite
sudo chmod -R g+rws /home/svn/myWebsite
sudo /etc/init.d/apache2 restart
sudo htpasswd /etc/subversion/passwd user1
sudo /etc/init.d/apache2 restart
All DONE.
Now you can access the SVN Server using….
http://uvo12exe3iancqqq622.vm.cld.sr/svn/myWebsite/
If you are still not able to open the url, that means you have IPtables services running and stopping you to access the server. For testing, you can stop the sudo ufw stop
To run single recipe using chef-solo
> chef-solo -c /opt/chef/repo/config/solo.rb -o my_cookbook::recipeTo run the single reciepe in same machine
> chef-apply hello.rb
To generate the cookbook standard structure
> chef generate cookbook learn_chef_httpd
> knife cookbook create smartmontools -r md
To generate the template file in cookbook
> chef generate template learn_chef_httpd index.html
To run the cookbook in local mode
> sudo chef-client --local-mode --runlist ‘recipe[learn_chef_httpd]’
To download the cookbooks from market place
> knife cookbook site download learn_chef_httpd
> knife cookbook site install learn_chef_httpd
To Upload the cookbooks to chef server
> knife cookbook upload learn_chef_httpd
> knife cookbook upload -a
To bootstrap a nodes
> knife bootstrap {{address}} --ssh-user {{user}} --ssh-password '{{password}}' --sudo --use-sudo-password --node-name node1 --run-list 'recipe[learn_chef_apache2]'
> knife bootstrap uvo1t75faaktzc532w6.vm.cld.sr -x root -P Br356YS0iy –sudo –node-name firefox
> knife bootstrap 123.45.6.789 -x username -P password –sudo
To see the list of nodes
>knife node list
To edit the node run list
> knife node edit name_of_node
To see the info about each nodes
> knife node show node1
Run the cooksbooks on nodes
> knife ssh {{address}} 'sudo chef-client' --manual-list --ssh-user {{user}} --ssh-password '{{password}}'
> ssh username@ipadddress -i mycredentials.pem sudo chef-client
To add the run list to the nodes.
knife node run_list add C2445575914.domain 'recipe[hptrain]'
knife node run_list set test-node '''recipe[iptables]''' [Windows - Powershell]
knife node run_list set test-node 'recipe[iptables]' [Windows - Command]
We do offer classroom training and workshop in Bangalore, Hyderabad, Pune, Mumbai and New Delhi. For more details, please send us an email to info@scmGalaxy.com
Also, you can follow this url Click Here
With Lifetime Enrollment
DevOps + Build & Release – INR 30K
Without Lifetime Enrollment
DevOps + Build & Release – INR 25K
Concept and Process
Operating Systems
Source Code Management
Build Tools
Scripting
Package Management in Linux and Windows
Artifact Repository Tools
Configuration Management Tools
Incident Management tools
Continuous Integration Tools
CI/CD Concept and Implementation
 |
 |
 |
 |
 |
 |
 |
 |
Way 1 –
Put following lines as part of Default Content text box
${BUILD_LOG, maxLines=9999, escapeHtml=false}
this works for free style projects
Way 2 – If you want send only specific line using regex
${BUILD_LOG_REGEX, regex=”\\b(Buildfile|BUILD)\\b”, linesAfter=1}
${BUILD_LOG_REGEX, regex=”^.*?BUILD FAILED.*?$”, linesBefore=0, linesAfter=10, maxMatches=5, showTruncatedLines=false, escapeHtml=true}
${BUILD_LOG_REGEX, regex=”\\b(FC0)\\b”, linesAfter=1}
${BUILD_LOG_REGEX, regex=”^.*?BUILD FAILED.*?$”, linesBefore=0, linesAfter=10, maxMatches=5, showTruncatedLines=false, escapeHtml=true}
${BUILD_LOG_REGEX, regex=”^.*?FC0.*?$”, maxMatches=95, showTruncatedLines=false, escapeHtml=false, matchedLineHtmlStyle=true}
This will show exactly which is expected lines as part of the regEx.
More Info
http://wiki.hudson-ci.org/display/HUDSON/Email-ext+plugin
http://champasoft.blogspot.nl/2012/11/jenkins-pattern-for-e-mailing-log.html
Solution 1
In Windows: Go to environment variable and add following variables
export JAVA_OPTS=”-XX:MaxPermSize=512m -Xms512m -Xmx1024m”
export GRADLE_OPTS=”-XX:MaxPermSize=512m”
export SONAR_RUNNER_OPTS=”-Xmx512m -XX:MaxPermSize=512m”
In Linux:
In my ~/.bash_profile file, I have set the following 3 variables. Note there are other variables in this file as well i.e. JAVA_HOME, ANT_HOME, SONAR_HOME, SONAR_RUNNER_HOME, etc:
export JAVA_OPTS=”-XX:MaxPermSize=512m -Xms512m -Xmx1024m”
export GRADLE_OPTS=”-XX:MaxPermSize=512m”
export SONAR_RUNNER_OPTS=”-Xmx512m -XX:MaxPermSize=512m”
Solution 2
You can set the -Xmx or -XX:MaxPermSize in your Jenkins global configuration. To do this, navigate to Jenkins global configuration page (Manage Jenkins -> Configure System)
Name – JAVA_OPTS
Value – “-XX:MaxPermSize=512m -Xms512m -Xmx1024m”
You can set similarlity for maven, gradle, sonar as well under “Maven Project Configuration” or grade project configuration.
Name – GRADLE_OPTS
Value – “-XX:MaxPermSize=512m”
Solution 3
They way you can modify this value depends on the distribution and depending on the slave type
Master
In RedHat, for example, you can do it under /etc/sysconfig/jenkins using the variable JENKINS_JAVA_OPTIONS… i.e JENKINS_JAVA_OPTIONS=”-XX:MaxPermSize=512m”.
In Ubuntu, the Jenkins configuration file is under /etc/default/jenkins and the variable to configure is JAVA_ARGS … i.e JAVA_ARGS=”-XX:MaxPermSize=512m”.
Slave
Sometimes slave machines will require permgen to be increased. Depending on the type of slave that is configured will require a different method for setting any java argument.
With SSH Slaves, the java process is started using an SSH command, so the “-XX:MaxPermSize=512m” can be set in the JVM Options in the Node configuration page.
With JNLP Slaves launching as a service on linux, you can add the VM arguments to the startup scripts (usually in /etc/sysconfig/jenkins or /etc/default/jenkins as part of the JAVA_ARGS variable)
With JNLP Slaves launching as a service on Windows requires modifications to be made on the jenkins.xml file which is located on slave machine in the JENKINS_HOME directory.
Reference
http://stackoverflow.com/questions/18000841/jenkins-java-lang-outofmemoryerror-permgen-space
https://wiki.jenkins-ci.org/display/JENKINS/Builds+failing+with+OutOfMemoryErrors
https://cloudbees.zendesk.com/hc/en-us/articles/204264000-Why-do-I-receive-java-lang-OutOfMemoryError-PermGen-space
http://stackoverflow.com/questions/14762162/how-do-i-give-jenkins-more-heap-space-when-its-running-as-a-daemon-on-ubuntu
http://stackoverflow.com/questions/18000841/jenkins-java-lang-outofmemoryerror-permgen-space
http://javarevisited.blogspot.nl/2011/09/javalangoutofmemoryerror-permgen-space.html
https://plumbr.eu/outofmemoryerror/permgen-space
Jenkins Troubleshooting
Jenkins is one of the important CI tools which many organization used as part of their Build and DevOps infrasture. I am going to consolidate all the troubshooting techniques which can be used in order to find any issues related in jenkins.
Step 1. Take a look at Log generated by the jenkins and plugins via http://jenkins-master/log
Note: Here you can also create your own logger and and set your custom log levels such as (severe, warning, info, config, file, finer, finest, all)
Step 2. Also, you can look at /var/log/jenkins/jenkins.log via doing “tail -f /var/log/jenkins/jenkins.log” and check carefully whats breaking it.
Log file in windows – jenkins.err.log & jenkins.out.log
Slave log file in windows – logs\slaves
Step 3. Check the system log file
Step 4. Few Jenkins plugins which can you help you
https://wiki.jenkins-ci.org/display/JENKINS/Monitoring
Step 4. Make sure your Jenkins & installed plugins version is installed with most uptodate stable release.
Some Common issues:
Issues 1. If Jenkins hangs, follow https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+is+hanging
Issues 2 Site is down with no 404 messages
Cause- Jenkins had an Out of Memory (OOM) error causing the server to kill the jenkins instance
Solution: Restart server; Review heap logs to identify root cause of error
Issues 3 Site is up however no jobs are displayed
Cause 1: AWS Server rebooted but data volume did not mount properly
Solution: mount <device> <path> on the instance
Issues 4 – java.lang.outofmemoryerror permgen space issues