Write 10 Chef Cookbooks to test your chef programming skills

chef-cookbooks

  1. Write a cookbook using you can install apache http server in CentOs and Ubuntu, enable the services and start the service.
  2. Write a recipe using template resources to create a file, but only if an attribute has a specific value.
    1. Write a recipe to create a file using a string, but not if “/etc/passwd” exists?
  3. Write a cookbook to unzip a file, and then move a files from one location to another using batch, bash chef resources?
  4. Create a role and assign any two cookbooks to role and setup a 2 nodes assigned to role and conerse the role.
  5. Write a cookbook using install version of latest tomcat and deploy jenkins.war files into war.
  6. Write a 4 cookbooks php, mysql, apache and webapp respectively and map a dependency between them and install a sample web application.
  7. Write a cookbook to install git, wget, zip in RHEL and Ubuntu.
  8. Write a cookbook in which create a file using a string, but only if “/etc/passwd” exists
  9. Write a coobook to run a batch file that unzips and then moves files from one location to another.
Tagged : / / / / / / / / / /

Configure the Knife Command – Chef

configure-the-knife-command

We now have to configure the knife command. This command is the central way of communicating with our server and the nodes that we will be configuring. We need to tell it how to authenticate and then generate a user to access the Chef server.

Luckily, we’ve been laying the groundwork for this step by acquiring the appropriate credential files. We can start the configuration by typing:

knife configure --initial 

This will ask you a series of questions. We will go through them one by one:

WARNING: No knife configuration file found Where should I put the config file? [/home/your_user/.chef/knife.rb]

The values in the brackets ([]) are the default values that knife will use if we do not select a value.

We want to place our knife configuration file in the hidden directory we have been using:

/home/your_user/chef-repo/.chef/knife.rb

In the next question, type in the domain name or IP address you use to access the Chef server. This should begin with https:// and end with :443:

https://server_domain_or_IP:443

You will be asked for a name for the new user you will be creating. Choose something descriptive:

Please enter a name for the new user: [root] station1

It will then ask you for the admin name. This you can just press enter on to accept the default value (we didn’t change the admin name).

It will then ask you for the location of the existing administrators key. This should be:

/home/your_user/chef-repo/.chef/admin.pem

It will ask a similar set of questions about the validator. We haven’t changed the validator’s name either, so we can keep that as chef-validator. Press enter to accept this value.

It will then ask you for the location of the validation key. It should be something like this:

/home/your_user/chef-repo/.chef/chef-validator.pem

Next, it will ask for the path to the repository. This is the chef-repo folder we have been operating in:

/home/your_user/chef-repo

Finally, it will ask you to select a password for your new user. Select anything you would like.

This should complete our knife configuration. If we look in our chef-repo/.chef directory, we should see a knife configuration file and the credentials of our new user:

ls ~/chef-repo/.chef 
admin.pem  chef-validator.pem  knife.rb  station1.pem
Tagged : / / / / / / / / / / / /

Chef solo | Chef solo Tutorial | Chef solo Guide

chef-solo
Install chef
============
> curl -L https://www.opscode.com/chef/install.sh | bash
Check
=====
> chef-solo -v
Setup chef repository
=====================
> wget http://github.com/opscode/chef-repo/tarball/master
> tar zxvf master
> mv opscode-chef-repo-f9d4b0c/ chef-repo
>ls chef-repo/
Create .chef directory inside chef-repo
=======================================
> mkdir chef-repo/.chef
Setup a local cookbook path
===========================
> vi chef-repo/.chef/knife.rb
cookbook_path [ ‘/root/chef-repo/cookbooks’ ]
Create your first cookbook
==========================
> knife cookbook create ntp
Writing your first recipe
=========================
> vi chef-repo/cookbooks/ntp/recipes/default.rb
package ‘ntp’
Now configure your server using chef-solo
=========================================
> vi chef-repo/solo.rb
file_cache_path “/root/chef-solo”
cookbook_path “/root/chef-repo/cookbooks”
> vi web.json
{
“run_list”: [ “recipe[ntp]” ]
}
Ensure ntp is not installed
===========================
> yum remove -y ntp
Use chef-solo to configure your server
=====================================
> chef-solo -c chef-repo/solo.rb -j chef-repo/web.json
Confirm ntp is installed
========================
> yum info ntp
Tagged : / / / / / /

Provision a AWS ec2 vm using chef | Step by Step Guide | AWS ec2 vm Tutorial

provision-a-aws-ec2-vm-using-chef

Provision a AWS ec2 vm using chef

Step 1: Install chefdk

Step 2: Setup AWS Credentails

Step X: Setup your knife config

Step X: Make sure following is set and exported in env.

 

AWS_ACCESS_KEY_ID=secrets AWS_SECRET_ACCESS_KEY=secrets AWS_DEFAULT_REGION=us-east-1 AWS_SSH_KEY=your_ssh_key_name AWS_ACCESS_KEY=secrets AWS_SECRET_KEY=secrets

 

Step 3: Genrate a new repository using the chef generate command

> chef generate repo chefdk-provision-demo
> cd chefdk-provision-demo

Step 4: Generate a provision cookbook. This is the required name, and it must be in the current directory.
> chef generate cookbook provision

Step 5: Edit the default recipe, $EDITOR provision/recipes/default.rb with following code…

context = ChefDK::ProvisioningData.context with_driver 'aws::us-west-2' options = { ssh_username: 'admin', use_private_ip_for_ssh: false, bootstrap_options: { key_name: 'jtimberman', image_id: 'ami-0d5b6c3d', instance_type: 'm3.medium', }, convergence_options: context.convergence_options, } machine context.node_name do machine_options options action context.action converge true end

Understand the code:
> To break this down, first we get the ChefDK provisioning context that will pass in options to chef-provisioning.
> Then we tell chef-provisioning to use the AWS driver, and in the us-west-2 region.
> The options hash is used to setup the instance.
> We’re using Debian 8, which uses the admin user to log in, an SSH key that exists in the AWS region, the actual AMI, and finally the instance type.
> Then, we’re going to set the convergence options automatically from ChefDK. This is the important part that will ensure the node has the right run list.

Step 6: Generate a Policyfile.rb and And edit its content, $EDITOR Policyfile.rb.
> chef generate policyfile
> vi policyfile.rb

name            "chefdk-provision-demo" default_source  :community run_list        "recipe[libuuid-user]" cookbook        "libuuid-user"

Here we’re simply getting the libuuid-user cookbook from Supermarket and applying the default recipe to the nodes that have this policy.

Step 7: The next step is to install the Policyfile. This generates the Policyfile.lock.json, and downloads the cookbooks to the cache, ~/.chefdk/cache/cookbooks. If this isn’t run, chef will complain, with a reminder to run it.

> chef install

Step 8: Finally, we can provision a testing system with this policy:

> chef provision testing –sync -n debian-libuuid

Reference:
http://jtimberman.housepub.org/blog/2015/05/15/quick-tip-chefdk-provision/

Tagged : / / / / / / / / / / / / / / / / /

How to install chefDK in RHEL, Ubantu, Mac and Windows?

chefdk-installtion-process-rhel-ubantu-mac-windows
How to install chefDK in RHEL, Ubantu, Mac and Windows?
How to install chefDK in RHEL
Step 1: Download checfdk from https://downloads.chef.io/chef-dk/
or
> chmod 755 chefdk-0.9.0-1.el6.x86_64.rpm
Step 2: Install chef dk
> rpm -ivh chefdk-0.9.0-1.el6.x86_64.rpm
Step 3: Confirm your installation.
> which chef
> chef -v
Install folder is – /opt/chefdk
Tagged : / / / / / / / / / / / / / / / / / / /

Chef Code Analysis using Foodcritic | Foodcritic Tutorial

chef-code-analysis-using-foodcritic

What is Foodcritic? Foodcritic is a static linting tool that analyzes all of the Ruby code that is authored in a cookbook against a number of rules, and then returns a list of violations. In another word, Foodcritic is a helpful lint tool you can use to check your Chef cookbooks for common problems.

We use Foodcritic to check cookbooks for common problems:
Style
Correctness
Syntax
Best practices
Common mistakes
Deprecations

Foodcritic does not
Foodcritic does not validate the intention of a recipe, rather it evaluates the structure of the code, and helps enforce specific behavior, detect portability of recipes, identify potential run-time failures, and spot common anti-patterns.

When Foodcritic returns a violation, this does not automatically mean the code needs to be changed. It is important to first understand the intention of the rule before making the changes it suggests.

Foodcritic has two goals:

To make it easier to flag problems in your Chef cookbooks that will cause Chef to blow up when you attempt to converge. This is about faster feedback. If you automate checks for common problems you can save a lot of time.

To encourage discussion within the Chef community on the more subjective stuff – what does a good cookbook look like? Opscode have avoided being overly prescriptive which by and large I think is a good thing. Having a set of rules to base discussion on helps drive out what we as a community think is good style.

Foodcritic built-in Rules
It comes with 47 built-in rules that identify problems ranging from simple style inconsistencies to difficult to diagnose issues that will hurt in production. If you want to see the list of rules, please navigate the url as below;
http://www.foodcritic.io/

Prerequisites
Foodcritic runs on Ruby (MRI) 1.9.2+ which depending on your workstation setup may be a more recent version of Ruby than you have installed. The Ruby Version Manager (RVM) is a popular choice for running multiple versions of ruby on the same workstation, so you can try foodcritic out without running the risk of damaging your main install

Foodcritic installation

Method 1
Install RVM as non-root user

$ sudo /etc/init.d/iptables stop OR sudo start ufw

$ curl -s raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer | bash -s stable
OR
$ sudo bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )
OR
$ curl -s raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer | sudo bash -s stable
OR
$ gpg –keyserver hkp://keys.gnupg.net –recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
OR
$ command curl -sSL https://rvm.io/mpapis.asc | gpg –import –

$ rvm get stable
$ rvm install ruby-2.2.3
$ gem install foodcritic

Method 2
Install ruby

$ sudo apt-get install ruby-2.2.3 (Ubantu)
$ sudo yum install ruby-2.2.3 (rhel)

Install foodcritic
> gem install foodcritic

Method 3
Alternatively install ChefDK which already includes foodcritic: https://downloads.getchef.com/chef-dk/

How to run Foodcritic?
You should now find you have a foodcritic command on your PATH. Run foodcritic to see what arguments it supports:

foodcritic [cookbook_path]
-r, –[no-]repl Drop into a REPL for interactive rule editing.
-t, –tags TAGS Only check against rules with the specified tags.
-f, –epic-fail TAGS Fail the build if any of the specified tags are matched.
-C, –[no-]context Show lines matched against rather than the default summary.
-I, –include PATH Additional rule file path(s) to load.
-S, –search-grammar PATH Specify grammar to use when validating search syntax.
-V, –version Display version.

How to setup Foodcritic with Jenkins

Configuring Jenkins to run foodcritic
To manually add a new job to Jenkins to check your cookbooks with foodcritic do the following:

  1. Ensure you have Ruby 1.9.2+ and the foodcritic gem installed on the box running Jenkins.
  2. You’ll probably need to install the Git plugin. In Jenkins select “Manage Jenkins” -> “Manage Plugins”. Select the “Available” tab. Check the checkbox next to the Git Plugin and click the “Install without restart” button.
  3. In Jenkins select “New Job”. Enter a name for the job “my-cookbook”, select “Build a free-style software project” and click “OK”.
  4. On the resulting page select “Git” under “Source Code Management” and enter the URL for your repo.
  5. Check the checkbox “Poll SCM” under “Build Triggers”.
  6. Click “Add Build Step” -> “Execute shell” under “Build”. This is where we will call foodcritic.
  7. Assuming you are using rvm enter the following as the command:
  8. #!/usr/bin/env rvm-shell 1.9.3
    foodcritic .
  9. Click “Save”.
  10. Cool, we’ve created your new job. Now lets see if it works. Click “Build Now” on the left-hand side.
  11. You can click the build progress bar to be taken directly to the console output.
  12. After a moment you should see that the build has been successful and foodcritic warnings (if any) are shown in your console output.
  13. Yes, for maximum goodness you should be automating all this with Chef. 🙂
  14. For more information refer to the instructions for building a “free-style software project” here:
    https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project
  15. See also this blog post about rvm-shell which ensures you have the right version of Ruby loaded when trying to build with foodcritic:
    http://blog.ninjahideout.com/posts/rvm-improved-support-for-hudson

Failing the build
The above is a start, but we’d also like to fail the build if there are any warnings that might stop the cookbook from working.

CI is only useful if people will act on it. Lets start by only failing the build when there is a correctness problem that would likely break our Chef run. We’ll continue to have the other warnings available for reference in the console log but only correctness issues will fail the build.

Select the “my-cookbook” job in Jenkins and click “Configure”.

Scroll down to our “Execute shell” command and change it to look like the following:

#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f correctness .
Click “Save” and then “Build Now”.

More complex expressions
Foodcritic supports more complex expressions with the standard Cucumber tag syntax. For example:

#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f any -f ~FC014 .
Here we use any to fail the build on any warning, but then use the tilde ~ to exclude FC014. The build will fail on any warning raised, except FC014.

You can find more detail on Cucumber tag expressions at the Cucumber wiki:

https://github.com/cucumber/cucumber/wiki/Tags

Tracking warnings over time
The Jenkins Warnings plugin can be configured to understand foodcritic output and track your cookbook warnings over time.

You’ll need to install the Warnings plugin. In Jenkins select “Manage Jenkins” -> “Manage Plugins”. Select the “Available” tab. Check the checkbox next to the Warnings Plugin and click the “Install without restart” button.

From “Manage Jenkins” select “Configure System”. Scroll down to the “Compiler Warnings” section and click the “Add” button next to “Parsers”.

Enter “Foodcritic” in the Name field.

Enter the following regex in the “Regular Expression” field:

^(FC[0-9]+): (.*): ([^:]+):([0-9]+)$

Enter the following Groovy script into the “Mapping Script” field:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(3)
String lineNumber = matcher.group(4)
String category = matcher.group(1)
String message = matcher.group(2)

return new Warning(fileName, Integer.parseInt(lineNumber), “Chef Lint Warning”, category, message);

To test the match, enter the following example message in the “Example Log Message” field:

FC001: Use strings in preference to symbols to access node attributes: ./recipes/innostore.rb:30
Click in the “Mapping Script” field and you should see the following appear below the Example Log Message:

One warning found
file name: ./recipes/innostore.rb
line number: 30
priority: Normal Priority
category: FC001
type: Chef Lint Warning
message: Use strings in prefe[…]ols to access node attributes
Cool, it’s parsed our example message successfully. Click “Save” to save the parser.

Select the “my-cookbook” job in Jenkins and click “Configure”.

Check the checkbox next to “Scan for compiler warnings” underneath “Post-build Actions”.

Click the “Add” button next to “Scan console log” and select our “Foodcritic” parser from the drop-down list.

Click the “Advanced…” button and check the “Run always” checkbox.

Click “Save” and then “Build Now”.

Add the bottom of the console log you should see something similar to this:

[WARNINGS] Parsing warnings in console log with parsers [Foodcritic]
[WARNINGS] Foodcritic : Found 48 warnings.
Click “Back to Project”. Once you have built the project a couple of times the warnings trend will appear here.

Reference:
http://acrmp.github.io/foodcritic/
https://docs.chef.io/foodcritic.html
http://www.foodcritic.io/
https://atom.io/packages/linter-foodcritic
http://www.slideshare.net/harthoover/rapid-chef-development-with-berkshelf-testkitchen-and-foodcritic

Tagged : / / / / / / / / / / /

Most Useful Tools for Chef

useful-tools-for-chef

Useful Tools for Chef 

Chef Code Correctness (Chef Code Analysis)

  • Foodcritic
  • Rubocop

Chef Unit Testing

  • ChefSpec

Chef Integration Testing

  • TestKitchen
  • ServerSpec
  • RSpec

Some other tools which can be used along with Chef

  • dsh
  • gsh

Some other chef & Knife tools

  • knife-lastrun
  • knife-preflight
  • Chef-handlers
  • knife-flip
  • knife-bulkchangeenvironment
  • knife-env-diff
  • knife spork

Chef Tools for Windows

  • POSHChef – POSHChef has been built as a native chef client on Windows using PowerShell.
Tagged : / / / / / / /

How to Implement Chef roles using Chef server ?

implement-chef-roles-using-chef-server

What is Role?

A role is a way to define certain patterns and processes that exist across nodes in an organization as belonging to a single job function. Each role consists of zero (or more) attributes and a run-list. Each node can have zero (or more) roles assigned to it. When a role is run against a node, the configuration details of that node are compared against the attributes of the role, and then the contents of that role’s run-list are applied to the node’s configuration details. When a chef-client runs, it merges its own attributes and run-lists with those contained within each assigned role.

How to use Roles in Chef?

  1. Create a Role and add the cookbooks into it.
  2. Assign the role into each node or bootstrap new nodes using roles
  3. The the run list

How to create Role?

Method 1: In Chef Server directly

> knife role create client1

&

Add the run list e.g. “recipe[nginx]” under “run_list”

Save & exit

The role will be created in Chef Server.

Example

{ "name": "client1", "description": "", "json_class": "Chef::Role", "default_attributes": { }, "override_attributes": { }, "chef_type": "role", "run_list": [ "recipe[nginx]", "recipe[phpapp::web]" ], "env_run_lists": { } } 

Let’s download the role from the Chef server so we have it locally in a Chef repository.

> knife role show client1 -d -Fjson > roles/client1.json

Now, Lets bootstrap the node using knife with roles

> knife bootstrap --run-list "role[client1]" --sudo hostname

How to edit the roles in chef Server?

> knife role edit client1

Method 2: In local repo under chef-repo folder

> vi webserver.rb

example –

name "web_server" description "Role for web servers" run_list(   "role[base]",   "recipe{web_server]" ) 

& Then upload to chef server using following commands.

> knife role from file path/to/role/file

How Assigning Roles to Nodes?

> knife node list
> knife node edit node_name

This will bring up the node’s definition file, which will allow us to add a role to its run_list:

{ "name": "client1", "chef_environment": "_default", "normal": { "tags": [  ] 	}, "run_list": [ 	"recipe[nginx]" ] } 

For instance, we can replace our recipe with our role in this file:

{ "name": "client1", "chef_environment": "_default", "normal": { "tags": [ ] }, "run_list": [ "role[web_server]" ] } 

How to bootstrap node using role?

> knife bootstrap {{address}} --ssh-user {{user}} --ssh-password '{{password}}' --sudo --use-sudo-password --node-name node1 --run-list 'role[production]'
> knife bootstrap --run-list "role[phpapp-web]" --sudo hostname

How to run roles against nodes?

You can run chef-client on multiple nodes via knife ssh command like, To query for all nodes that have the webserver role and then use SSH to run the command sudo chef-client, enter:

> knife ssh "role:webserver" "sudo chef-client"

To find the uptime of all of web servers running Ubuntu on the Amazon EC2 platform, enter:

> knife ssh "role:web" "uptime" -x ubuntu -a ec2.public_hostname

Reference

http://docs.chef.io/roles.html

https://docs.chef.io/knife_ssh.html

https://docs.chef.io/knife_role.html

Tagged : / / / / / / / / / / / / /

Chef Commands Line Reference | Chef Commands Line Guide | Cheatsheet

chef-commands-line-reference

To run single recipe using chef-solo

> chef-solo -c /opt/chef/repo/config/solo.rb -o my_cookbook::recipe

To run the single reciepe in same machine

> chef-apply hello.rb

To generate the cookbook standard structure

> chef generate cookbook learn_chef_httpd
> knife cookbook create smartmontools -r md

To generate the template file in cookbook

> chef generate template learn_chef_httpd index.html

To run the cookbook in local mode

> sudo chef-client --local-mode --runlist ‘recipe[learn_chef_httpd]’

To download the cookbooks from market place

> knife cookbook site download learn_chef_httpd
> knife cookbook site install learn_chef_httpd

To Upload the cookbooks to chef server

> knife cookbook upload learn_chef_httpd
> knife cookbook upload -a

To bootstrap a nodes

> knife bootstrap {{address}} --ssh-user {{user}} --ssh-password '{{password}}' --sudo --use-sudo-password --node-name node1 --run-list 'recipe[learn_chef_apache2]'
> knife bootstrap uvo1t75faaktzc532w6.vm.cld.sr -x root -P Br356YS0iy –sudo –node-name firefox
> knife bootstrap 123.45.6.789 -x username -P password –sudo

To see the list of nodes

>knife node list

To edit the node run list

> knife node edit name_of_node

To see the info about each nodes

> knife node show node1

Run the cooksbooks on nodes

> knife ssh {{address}} 'sudo chef-client' --manual-list --ssh-user {{user}} --ssh-password '{{password}}'
> ssh username@ipadddress -i mycredentials.pem sudo chef-client

To add the run list to the nodes.

knife node run_list add C2445575914.domain 'recipe[hptrain]'
knife node run_list set test-node '''recipe[iptables]''' [Windows - Powershell]
knife node run_list set test-node 'recipe[iptables]' [Windows - Command]
Tagged : / / / / / / / / / / / /

Chef – Documenting Cookbooks automatically

chef-documenting-cookbooks

Problems Area –

Our infrastructure has many cookbooks that aim to be reusable, primarily through encapsulating behaviour in LWRPs. This led to an explosion of LWRPs and sometimes the documentation didn’t keep up or did just not exist.

Solution 1: Follow Best Practices 

This command will create a README.rdoc by default, and I prefer Markdown, so I specify the -r md option.

> knife cookbook create smartmontools -r md

Solution 2: Automate using plugins

README.md which should be generated automatcially must contain information about the recipes, attributes, platform compatibility and cookbook requirements (i.e. depends, recommends, suggests etc).

Mathias Lafeldt wrote a knife plugin that generates an initial README.md from the metadata.rb file in a a cookbook.

Link –

Extension of Mathias Lafeldt was done here with regenerate README.md from the cookbook source code

Please refer these useful blogs as well which is referenced in order to write this article…

Tagged : / / / / / / / / / / / /