Ecosystem of chef and Its associated tools explained

Chef Apply
chef-apply is an executable program that runs a single recipe from the command line. Is part of the Chef development kit. A great way to explore resources

Chef
The chef executable is a command-line tool which Generates applications, cookbooks, recipes, attributes, files, templates, and custom resources (LWRPs) and Ensures that RubyGems are downloaded properly for the chef-client development environment along with Verifies that all components are installed and configured correctly

Knife
knife is a command-line tool that provides an interface between a local chef-repo and the Chef server. knife helps users to manage Nodes, Cookbooks and recipes, Roles, Environments, and Data Bags, Resources within various cloud environments, The installation of the chef-client onto nodes, Searching of indexed data on the Chef server

Chef Client
The Chef client works with the Chef server to bring nodes to their desired states with policies you provide as recipes. The chef-client executable can be run as a daemon. A chef-client is an agent that runs locally on every node that is under management by Chef. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including:

  • Registering and authenticating the node with the Chef server
  • Building the node object
  • Synchronizing cookbooks
  • Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and
  • all other dependencies
  • Taking the appropriate and required actions to configure the node
  • Looking for exceptions and notifications, handling each as required

Chef Development Kit
The Chef development kit contains all you need to develop and test your infrastructure, built by the awesome Chef community. Chef Development Kit has following Component installed…

  • fauxhai
  • kitchen-vagrant
  • openssl
  • delivery-cli
  • test-kitchen
  • git
  • berkshelf
  • chefspec
  • knife-spork
  • inspec
  • tk-policyfile-provisioner
  • opscode-pushy-client
  • chef-dk
  • chef-sugar
  • chef-client
  • generated-cookbooks-pass-chefspec
  • chef-provisioning
  • package installation

Chef Server
The Chef server makes it easy to automate your infrastructure, manage scale and complexity, and safeguard your systems.

Chef Server has following tools which should be running…

  • bookshelf
  • nginx
  • oc_bifrost
  • oc_id
  • opscode-erchef
  • opscode-expander
  • opscode-solr4
  • postgresql
  • rabbitmq
  • redis_lb

InSpec
InSpec is an open-source testing framework for infrastructure with a human- and machine-readable language for specifying compliance, security and policy requirements.

Push Jobs Client
The Push Jobs client communicates with the Push Jobs server, which extends the Chef Server to allow you to execute commands across hundreds or even thousands of nodes in your Chef-managed infrastructure.

Push Jobs Server
The Push Jobs server add-on, along with its associated client, extends the Chef Server to allow you to execute commands across hundreds or even thousands of nodes in your Chef-managed infrastructure.

Supermarket
Supermarket is an artifact repository that makes it easy to browse, use, and share communal cookbooks and tools within your organization.

Chef Automate
One platform with a unified workflow, end-to-end visibility, and automated compliance over your entire Chef ecosystem.

Chef Compliance
Assess and monitor infrastructure compliance and use InSpec compliance profiles to validate that production servers are properly configured.

Chef Backend
Chef High Availability makes it easy to build high-availability Chef clusters on any infrastructure.

Chef Manage
Chef Manage is an Enterprise Chef add-on that enables a web-based user interface for visualizing and managing nodes, data bags, roles, environments, cookbooks and role-based access control (RBAC).

Kitchen or Test Kitchen
kitchen is the command-line tool for Kitchen, an integration testing tool used by the chef-client. Kitchen runs tests against any combination of platforms using any combination of test suites. Each test, however, is done against a specific instance, which is comprised of a single platform and a single set of testing criteria.

“Test Kitchen is an integration tool for developing and testing infrastructure code and software on isolated target platforms.” It creates test machines, converges them, and runs post-convergence tests against them to verify their state. Test Kitchen is written in Ruby. It has a plugin system for supporting machine creation through a variety of virtual machine technologies such as vagrant, EC2, docker, and several others. Test Kitchen makes it easy for Chef developers to test cookbooks on a variety of platforms. It uses busser to install post-convergence integration test tools such as Serverspec or BATS that actually perform the tests.

foodcritic
Foodcritic is a helpful lint tool you can use to check your Chef cookbooks for common problems.
http://www.foodcritic.io/

ChefSpec
ChefSpec is a framework that tests resources and recipes as part of a simulated chef-client run. ChefSpec tests execute very quickly. When used as part of the cookbook authoring workflow, ChefSpec tests are often the first indicator of problems that may exist within a cookbook.
ChefSpec is packaged as part of the Chef development kit. To run ChefSpec
$ chef exec rspec
https://docs.chef.io/chefspec.html

RuboCop
Rubocop is a Ruby command-line tool that performs lint and style checks based on the community driven Ruby Style Guide. It performs static analysis of any Ruby code, which includes Chef recipes, resources, library helpers, and so forth. Rubocop can be configured via .rubocop.yml to exclude certain rules, and it can be run with “–lint” to perform only lint checking, excluding all style checks. Rubocop is used in the Chef community in cookbooks to make contributions more consistent and easier to manage.

Serverspec
Serverspec is an “outside-in” integration test framework. It is platform and tool agnostic, and is used by other configuration management systems to verify systems are configured as desired. It checks the actual state of the target node by executing commands locally, via SSH, via WinRM, or other remote transports. Serverspec is implemented in RSpec, and uses RSpec test syntax.

Tagged : / / / / / / / / / / / / / / / / / / / /

Chef Code Analysis using Foodcritic | Foodcritic Tutorial

chef-code-analysis-using-foodcritic

What is Foodcritic? Foodcritic is a static linting tool that analyzes all of the Ruby code that is authored in a cookbook against a number of rules, and then returns a list of violations. In another word, Foodcritic is a helpful lint tool you can use to check your Chef cookbooks for common problems.

We use Foodcritic to check cookbooks for common problems:
Style
Correctness
Syntax
Best practices
Common mistakes
Deprecations

Foodcritic does not
Foodcritic does not validate the intention of a recipe, rather it evaluates the structure of the code, and helps enforce specific behavior, detect portability of recipes, identify potential run-time failures, and spot common anti-patterns.

When Foodcritic returns a violation, this does not automatically mean the code needs to be changed. It is important to first understand the intention of the rule before making the changes it suggests.

Foodcritic has two goals:

To make it easier to flag problems in your Chef cookbooks that will cause Chef to blow up when you attempt to converge. This is about faster feedback. If you automate checks for common problems you can save a lot of time.

To encourage discussion within the Chef community on the more subjective stuff – what does a good cookbook look like? Opscode have avoided being overly prescriptive which by and large I think is a good thing. Having a set of rules to base discussion on helps drive out what we as a community think is good style.

Foodcritic built-in Rules
It comes with 47 built-in rules that identify problems ranging from simple style inconsistencies to difficult to diagnose issues that will hurt in production. If you want to see the list of rules, please navigate the url as below;
http://www.foodcritic.io/

Prerequisites
Foodcritic runs on Ruby (MRI) 1.9.2+ which depending on your workstation setup may be a more recent version of Ruby than you have installed. The Ruby Version Manager (RVM) is a popular choice for running multiple versions of ruby on the same workstation, so you can try foodcritic out without running the risk of damaging your main install

Foodcritic installation

Method 1
Install RVM as non-root user

$ sudo /etc/init.d/iptables stop OR sudo start ufw

$ curl -s raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer | bash -s stable
OR
$ sudo bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )
OR
$ curl -s raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer | sudo bash -s stable
OR
$ gpg –keyserver hkp://keys.gnupg.net –recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
OR
$ command curl -sSL https://rvm.io/mpapis.asc | gpg –import –

$ rvm get stable
$ rvm install ruby-2.2.3
$ gem install foodcritic

Method 2
Install ruby

$ sudo apt-get install ruby-2.2.3 (Ubantu)
$ sudo yum install ruby-2.2.3 (rhel)

Install foodcritic
> gem install foodcritic

Method 3
Alternatively install ChefDK which already includes foodcritic: https://downloads.getchef.com/chef-dk/

How to run Foodcritic?
You should now find you have a foodcritic command on your PATH. Run foodcritic to see what arguments it supports:

foodcritic [cookbook_path]
-r, –[no-]repl Drop into a REPL for interactive rule editing.
-t, –tags TAGS Only check against rules with the specified tags.
-f, –epic-fail TAGS Fail the build if any of the specified tags are matched.
-C, –[no-]context Show lines matched against rather than the default summary.
-I, –include PATH Additional rule file path(s) to load.
-S, –search-grammar PATH Specify grammar to use when validating search syntax.
-V, –version Display version.

How to setup Foodcritic with Jenkins

Configuring Jenkins to run foodcritic
To manually add a new job to Jenkins to check your cookbooks with foodcritic do the following:

  1. Ensure you have Ruby 1.9.2+ and the foodcritic gem installed on the box running Jenkins.
  2. You’ll probably need to install the Git plugin. In Jenkins select “Manage Jenkins” -> “Manage Plugins”. Select the “Available” tab. Check the checkbox next to the Git Plugin and click the “Install without restart” button.
  3. In Jenkins select “New Job”. Enter a name for the job “my-cookbook”, select “Build a free-style software project” and click “OK”.
  4. On the resulting page select “Git” under “Source Code Management” and enter the URL for your repo.
  5. Check the checkbox “Poll SCM” under “Build Triggers”.
  6. Click “Add Build Step” -> “Execute shell” under “Build”. This is where we will call foodcritic.
  7. Assuming you are using rvm enter the following as the command:
  8. #!/usr/bin/env rvm-shell 1.9.3
    foodcritic .
  9. Click “Save”.
  10. Cool, we’ve created your new job. Now lets see if it works. Click “Build Now” on the left-hand side.
  11. You can click the build progress bar to be taken directly to the console output.
  12. After a moment you should see that the build has been successful and foodcritic warnings (if any) are shown in your console output.
  13. Yes, for maximum goodness you should be automating all this with Chef. 🙂
  14. For more information refer to the instructions for building a “free-style software project” here:
    https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project
  15. See also this blog post about rvm-shell which ensures you have the right version of Ruby loaded when trying to build with foodcritic:
    http://blog.ninjahideout.com/posts/rvm-improved-support-for-hudson

Failing the build
The above is a start, but we’d also like to fail the build if there are any warnings that might stop the cookbook from working.

CI is only useful if people will act on it. Lets start by only failing the build when there is a correctness problem that would likely break our Chef run. We’ll continue to have the other warnings available for reference in the console log but only correctness issues will fail the build.

Select the “my-cookbook” job in Jenkins and click “Configure”.

Scroll down to our “Execute shell” command and change it to look like the following:

#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f correctness .
Click “Save” and then “Build Now”.

More complex expressions
Foodcritic supports more complex expressions with the standard Cucumber tag syntax. For example:

#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f any -f ~FC014 .
Here we use any to fail the build on any warning, but then use the tilde ~ to exclude FC014. The build will fail on any warning raised, except FC014.

You can find more detail on Cucumber tag expressions at the Cucumber wiki:

https://github.com/cucumber/cucumber/wiki/Tags

Tracking warnings over time
The Jenkins Warnings plugin can be configured to understand foodcritic output and track your cookbook warnings over time.

You’ll need to install the Warnings plugin. In Jenkins select “Manage Jenkins” -> “Manage Plugins”. Select the “Available” tab. Check the checkbox next to the Warnings Plugin and click the “Install without restart” button.

From “Manage Jenkins” select “Configure System”. Scroll down to the “Compiler Warnings” section and click the “Add” button next to “Parsers”.

Enter “Foodcritic” in the Name field.

Enter the following regex in the “Regular Expression” field:

^(FC[0-9]+): (.*): ([^:]+):([0-9]+)$

Enter the following Groovy script into the “Mapping Script” field:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(3)
String lineNumber = matcher.group(4)
String category = matcher.group(1)
String message = matcher.group(2)

return new Warning(fileName, Integer.parseInt(lineNumber), “Chef Lint Warning”, category, message);

To test the match, enter the following example message in the “Example Log Message” field:

FC001: Use strings in preference to symbols to access node attributes: ./recipes/innostore.rb:30
Click in the “Mapping Script” field and you should see the following appear below the Example Log Message:

One warning found
file name: ./recipes/innostore.rb
line number: 30
priority: Normal Priority
category: FC001
type: Chef Lint Warning
message: Use strings in prefe[…]ols to access node attributes
Cool, it’s parsed our example message successfully. Click “Save” to save the parser.

Select the “my-cookbook” job in Jenkins and click “Configure”.

Check the checkbox next to “Scan for compiler warnings” underneath “Post-build Actions”.

Click the “Add” button next to “Scan console log” and select our “Foodcritic” parser from the drop-down list.

Click the “Advanced…” button and check the “Run always” checkbox.

Click “Save” and then “Build Now”.

Add the bottom of the console log you should see something similar to this:

[WARNINGS] Parsing warnings in console log with parsers [Foodcritic]
[WARNINGS] Foodcritic : Found 48 warnings.
Click “Back to Project”. Once you have built the project a couple of times the warnings trend will appear here.

Reference:
http://acrmp.github.io/foodcritic/
https://docs.chef.io/foodcritic.html
http://www.foodcritic.io/
https://atom.io/packages/linter-foodcritic
http://www.slideshare.net/harthoover/rapid-chef-development-with-berkshelf-testkitchen-and-foodcritic

Tagged : / / / / / / / / / / /