Chef Node Objects Defined!

Chef Node Objects Defined!

For the chef-client, two important aspects of nodes are groups of attributes and run-lists.

  1. An attribute is a specific piece of data about the node, such as a network interface, a file system, the number of clients a service running on a node is capable of accepting, and so on.
  2. A run-list is an ordered list of recipes and/or roles that are run in an exact order.

The node object consists of the run-list and node attributes, which is a JSON file that is stored on the Chef server. The chef-client gets a copy of the node object from the Chef server during each chef-client run and places an updated copy on the Chef server at the end of each chef-client run.

An attribute is a specific detail about a node. Attributes are used by the chef-client to understand:

  • The current state of the node
  • What the state of the node was at the end of the previous chef-client run
  • What the state of the node should be at the end of the current chef-client run

Attributes are defined by:

  • The state of the node itself
  • Cookbooks (in attribute files and/or recipes)
  • Roles
  • Environments

During every chef-client run, the chef-client builds the attribute list using:

  • Data about the node collected by Ohai
  • The node object that was saved to the Chef server at the end of the previous chef-client run
  • The rebuilt node object from the current chef-client run, after it is updated for changes to cookbooks (attribute files and/or recipes), roles, and/or environments, and updated for any changes to the state of the node itself

After the node object is rebuilt, all of the attributes are compared, and then the node is updated based on attribute precedence. At the end of every chef-client run, the node object that defines the current state of the node is uploaded to the Chef server so that it can be indexed for search.

Tagged : /

Understand the chefignore file

The chefignore file is used to tell knife which cookbook files in the chef-repo should be ignored when uploading data to the Chef server. The type of data that should be ignored includes swap files, version control data, build output data, and so on. .

The chefignore file can be located in any subdirectory of a chef-repo: /, /cookbooks, /cookbooks/COOKBOOK_NAME/, roles, etc. It should contain sections similar to the following:

Example of chefignore


# Ignore editor swap files
*~
*.sw[a-z]

# Ignore top-level Subversion data
*/.svn/*

# Ignore all files in a directory
files/default/subdirectory/*
files/default/subdirectory/**

# OS generated files #
######################
.DS_Store
Icon?
nohup.out
ehthumbs.db
Thumbs.db

# SASS #
########
.sass-cache

# EDITORS #
###########
\#*
.#*
*~
*.sw[a-z]
*.bak
REVISION
TAGS*
tmtags
*_flymake.*
*_flymake
*.tmproj
.project
.settings
mkmf.log

## COMPILED ##
##############
a.out
*.o
*.pyc
*.so
*.com
*.class
*.dll
*.exe
*/rdoc/

# Testing #
###########
.watchr
.rspec
spec/*
spec/fixtures/*
test/*
features/*
examples/*
Guardfile
Procfile
.kitchen*
.rubocop.yml

# SCM #
#######
.git
*/.git
.gitignore
.gitmodules
.gitconfig
.gitattributes
.svn
*/.bzr/*
*/.hg/*
*/.svn/*

# Berkshelf #
#############
Berksfile
Berksfile.lock
cookbooks/*
tmp

# Cookbooks #
#############
CONTRIBUTING*
CHANGELOG*
TESTING*

# Strainer #
############
Colanderfile
Strainerfile
.colander
.strainer

# Vagrant #
###########
.vagrant
Vagrantfile

# Travis #
##########
.travis.yml

Reference 

https://docs.chef.io/chef_repo.html#chefignore-files

Tagged : /

run_list and env_run_list explanined in chef role

run-list
A run-list defines all of the information necessary for Chef to configure a node into the desired state. A run-list may include roles and/or recipes.

A run-list must be in one of the following formats: fully qualified, cookbook, or default. Both roles and recipes must be in quotes, for example:

'role[NAME]'
'recipe[COOKBOOK::RECIPE]'
'recipe[COOKBOOK::RECIPE],COOKBOOK::RECIPE,role[NAME]'

env_run_list
When you specify the env_run_list in role and the chef-client will pick up the _default run-list if env_run_list[environment_name] is null or nonexistent. env_run_list is only for roles.The env run list is an override, so it totally replaces the default non-env run list if the env matches.

{
 "name": "webserver",
 "default_attributes": {
 },
 "json_class": "Chef::Role",
 "env_run_lists": {
 "production": [],
 "preprod": [],
 "test": [ "role[base]", "recipe[apache]" "recipe[apache::copy_test_configs]" ],
 "dev": [ "role[base]", "recipe[apache]", "recipe[apache::copy_dev_configs]" ]
 },
 "run_list": [ "role[base]", "recipe[apache]" ],
 "description": "The webserver role",
 "chef_type": "role",
 "override_attributes": {
 }
}

where:

  • “webserver” is the name of the role
  • “env_run_lists” is a hash of per-environment run-lists for production, preprod, test, and dev
  • “production” and “preprod” use the default run-list because they do not have a per-environment run-list
  • “run_list” defines the default run-list

Reference
https://docs.chef.io/run_lists.html
https://railsware.com/blog/2013/09/26/getting-started-with-chef-server-part-2/

Tagged : / / /

How to run chef-client in why-run mode aka “no-operation”

why-run mode is a way to see what the chef-client would have configured, had an actual chef-client run occurred. This approach is similar to the concept of “no-operation” (or “no-op”): decide what should be done, but then don’t actually do anything until it’s done right. Run the executable in why-run mode, which is a type of chef-client run that does everything except modify the system.

When why-run mode is enabled, a chef-client run will occur that does everything up to the point at which configuration would normally occur. This includes

  1. Getting the configuration data,
  2. Authenticating to the Chef server,
  3. Rebuilding the node object,
  4. Expanding the run-list,
  5. Getting the necessary cookbook files,
  6. Resetting node attributes,
  7. Identifying the resources, and
  8. Building the resource collection and
  9. Does not include mapping each resource to a provider or configuring any part of the system.

How to run chef-client?

$ sudo chef-client -o 'recipe[ntp]' --why-run
$ sudo chef-client -o 'recipe[ntp]' -W

Tagged : / / /

Ecosystem of chef and Its associated tools explained

Chef Apply
chef-apply is an executable program that runs a single recipe from the command line. Is part of the Chef development kit. A great way to explore resources

Chef
The chef executable is a command-line tool which Generates applications, cookbooks, recipes, attributes, files, templates, and custom resources (LWRPs) and Ensures that RubyGems are downloaded properly for the chef-client development environment along with Verifies that all components are installed and configured correctly

Knife
knife is a command-line tool that provides an interface between a local chef-repo and the Chef server. knife helps users to manage Nodes, Cookbooks and recipes, Roles, Environments, and Data Bags, Resources within various cloud environments, The installation of the chef-client onto nodes, Searching of indexed data on the Chef server

Chef Client
The Chef client works with the Chef server to bring nodes to their desired states with policies you provide as recipes. The chef-client executable can be run as a daemon. A chef-client is an agent that runs locally on every node that is under management by Chef. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including:

  • Registering and authenticating the node with the Chef server
  • Building the node object
  • Synchronizing cookbooks
  • Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and
  • all other dependencies
  • Taking the appropriate and required actions to configure the node
  • Looking for exceptions and notifications, handling each as required

Chef Development Kit
The Chef development kit contains all you need to develop and test your infrastructure, built by the awesome Chef community. Chef Development Kit has following Component installed…

  • fauxhai
  • kitchen-vagrant
  • openssl
  • delivery-cli
  • test-kitchen
  • git
  • berkshelf
  • chefspec
  • knife-spork
  • inspec
  • tk-policyfile-provisioner
  • opscode-pushy-client
  • chef-dk
  • chef-sugar
  • chef-client
  • generated-cookbooks-pass-chefspec
  • chef-provisioning
  • package installation

Chef Server
The Chef server makes it easy to automate your infrastructure, manage scale and complexity, and safeguard your systems.

Chef Server has following tools which should be running…

  • bookshelf
  • nginx
  • oc_bifrost
  • oc_id
  • opscode-erchef
  • opscode-expander
  • opscode-solr4
  • postgresql
  • rabbitmq
  • redis_lb

InSpec
InSpec is an open-source testing framework for infrastructure with a human- and machine-readable language for specifying compliance, security and policy requirements.

Push Jobs Client
The Push Jobs client communicates with the Push Jobs server, which extends the Chef Server to allow you to execute commands across hundreds or even thousands of nodes in your Chef-managed infrastructure.

Push Jobs Server
The Push Jobs server add-on, along with its associated client, extends the Chef Server to allow you to execute commands across hundreds or even thousands of nodes in your Chef-managed infrastructure.

Supermarket
Supermarket is an artifact repository that makes it easy to browse, use, and share communal cookbooks and tools within your organization.

Chef Automate
One platform with a unified workflow, end-to-end visibility, and automated compliance over your entire Chef ecosystem.

Chef Compliance
Assess and monitor infrastructure compliance and use InSpec compliance profiles to validate that production servers are properly configured.

Chef Backend
Chef High Availability makes it easy to build high-availability Chef clusters on any infrastructure.

Chef Manage
Chef Manage is an Enterprise Chef add-on that enables a web-based user interface for visualizing and managing nodes, data bags, roles, environments, cookbooks and role-based access control (RBAC).

Kitchen or Test Kitchen
kitchen is the command-line tool for Kitchen, an integration testing tool used by the chef-client. Kitchen runs tests against any combination of platforms using any combination of test suites. Each test, however, is done against a specific instance, which is comprised of a single platform and a single set of testing criteria.

“Test Kitchen is an integration tool for developing and testing infrastructure code and software on isolated target platforms.” It creates test machines, converges them, and runs post-convergence tests against them to verify their state. Test Kitchen is written in Ruby. It has a plugin system for supporting machine creation through a variety of virtual machine technologies such as vagrant, EC2, docker, and several others. Test Kitchen makes it easy for Chef developers to test cookbooks on a variety of platforms. It uses busser to install post-convergence integration test tools such as Serverspec or BATS that actually perform the tests.

foodcritic
Foodcritic is a helpful lint tool you can use to check your Chef cookbooks for common problems.
http://www.foodcritic.io/

ChefSpec
ChefSpec is a framework that tests resources and recipes as part of a simulated chef-client run. ChefSpec tests execute very quickly. When used as part of the cookbook authoring workflow, ChefSpec tests are often the first indicator of problems that may exist within a cookbook.
ChefSpec is packaged as part of the Chef development kit. To run ChefSpec
$ chef exec rspec
https://docs.chef.io/chefspec.html

RuboCop
Rubocop is a Ruby command-line tool that performs lint and style checks based on the community driven Ruby Style Guide. It performs static analysis of any Ruby code, which includes Chef recipes, resources, library helpers, and so forth. Rubocop can be configured via .rubocop.yml to exclude certain rules, and it can be run with “–lint” to perform only lint checking, excluding all style checks. Rubocop is used in the Chef community in cookbooks to make contributions more consistent and easier to manage.

Serverspec
Serverspec is an “outside-in” integration test framework. It is platform and tool agnostic, and is used by other configuration management systems to verify systems are configured as desired. It checks the actual state of the target node by executing commands locally, via SSH, via WinRM, or other remote transports. Serverspec is implemented in RSpec, and uses RSpec test syntax.

Tagged : / / / / / / / / / / / / / / / / / / / /

How to install Chef Development Kit(ChefDK)

How to install Chef Development Kit(ChefDK)?

The Chef development kit contains all you need to develop and test your infrastructure, built by the awesome Chef community. Its Chef Developer tool.

Platform – Linux RHEL

Download Chef Development Kit(ChefDK)

# Download Package URL from https://downloads.chef.io/chefdk/2.5.3
$ sudo -s
$ cd /opt/
$ yum install wget -y
$ wget https://packages.chef.io/files/stable/chefdk/2.5.3/el/7/chefdk-2.5.3-1.el7.x86_64.rpm

Install Chef Development Kit(ChefDK)

$ rpm -Uvh chefdk-2.5.3-1.el7.x86_64.rpm

warning: chefdk-2.5.3-1.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:chefdk-2.5.3-1.el7               ################################# [100%]
Thank you for installing Chef Development Kit!

Verify Chef Development Kit(ChefDK) Quick way

$ chef -v
Chef Development Kit Version: 2.5.3
chef-client version: 13.8.5
delivery version: master (73ebb72a6c42b3d2ff5370c476be800fee7e5427)
berks version: 6.3.1
kitchen version: 1.20.0
inspec version: 1.51.21

Verify Chef Development Kit(ChefDK) Detailed way

$ chef verify
[WARN] This is an internal command used by the ChefDK development team. If you are a ChefDK user, please do not run it.
Running verification for component 'berkshelf'
Running verification for component 'test-kitchen'
Running verification for component 'tk-policyfile-provisioner'
Running verification for component 'chef-client'
Running verification for component 'chef-dk'
Running verification for component 'chef-provisioning'
Running verification for component 'chefspec'
Running verification for component 'generated-cookbooks-pass-chefspec'
Running verification for component 'fauxhai'
Running verification for component 'knife-spork'
Running verification for component 'kitchen-vagrant'
Running verification for component 'package installation'
Running verification for component 'openssl'
Running verification for component 'inspec'
Running verification for component 'delivery-cli'
Running verification for component 'git'
Running verification for component 'opscode-pushy-client'
Running verification for component 'chef-sugar'
.................................../opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/chef-provisioning-aws-3.0.2/lib/chef/resource/aws_route53_record_set.rb:48: warning: constant ::Fixnum is deprecated
cannot load such file -- fog
.......
---------------------------------------------
Verification of component 'fauxhai' succeeded.
Verification of component 'kitchen-vagrant' succeeded.
Verification of component 'openssl' succeeded.
Verification of component 'delivery-cli' succeeded.
Verification of component 'test-kitchen' succeeded.
Verification of component 'git' succeeded.
Verification of component 'berkshelf' succeeded.
Verification of component 'chefspec' succeeded.
Verification of component 'knife-spork' succeeded.
Verification of component 'inspec' succeeded.
Verification of component 'tk-policyfile-provisioner' succeeded.
Verification of component 'opscode-pushy-client' succeeded.
Verification of component 'chef-dk' succeeded.
Verification of component 'chef-sugar' succeeded.
Verification of component 'chef-client' succeeded.
Verification of component 'generated-cookbooks-pass-chefspec' succeeded.
Verification of component 'chef-provisioning' succeeded.
Verification of component 'package installation' succeeded.

 

Tagged : / /