## Logging in Chef Explained

Chef Server
All logs generated by the Chef server can be found in /var/log/opscode. Each service enabled on the system also has a sub-directory in which service-specific logs are located, typically found in /var/log/opscode/service_name.

The Chef server has built-in support for easily tailing the logs that are generated. To view all the logs being generated on the Chef server, enter the following command:
> chef-server-ctl tail

To view logs for a specific service:
> chef-server-ctl tail SERVICENAME
where SERVICENAME should be replaced with name of the service for which log files will be viewed. SERVICENAME represents the name of any service that is listed after running the “> chef-server-ctl service-list” subcommand.

Another way to view log files is to use the system utility tail:
> tail -50f /var/log/chef-server/opscode-chef/current
> tail -50f /var/log/opscode/opscode-chef/current

Supervisor Logs
Supervisor logs are created and managed directly by the service supervisor, and are automatically rotated when a the current log file reaches 1,000,000 bytes. 10 log files are kept. The latest supervisor log is always located in /var/log/chef-server/service_name/current and rotated logs have a filename starting with @ followed by a precise tai64n timestamp based on when the file was rotated.

Supervisor logs are available for the following services:

1. bifrost
2. bookshelf
3. nginx
4. opscode-erchef
5. opscode-expander
6. opscode-expander-reindexer
7. opscode-solr4
8. postgresql
9. rabbitmq
10. redis

### nginx

The nginx service creates both supervisor and administrator logs. The administrator logs contain both access and error logs for each virtual host utilized by the Chef server. Each of the following logs require external log rotation.

Logs Description
/var/log/opscode/nginx/access.log The Web UI and API HTTP access logs.
/var/log/opscode/nginx/error.log The Web UI and API HTTP error logs.
/var/log/opscode/nginx/internal-account.access.log The opscode-account internal load-balancer access logs.
/var/log/opscode/nginx/internal-account.error.log The opscode-account internal load-balancer error logs.
/var/log/opscode/nginx/internal-authz.access.log The opscode-authz internal load-balancer access logs.
/var/log/opscode/nginx/internal-authz.error.log The opscode-authz internal load-balancer error logs.
/var/log/opscode/nginx/internal-chef.access.log The opscode-chef and opscode-erchef internal load-balancer access logs.
/var/log/opscode/nginx/internal-chef.error.log The opscode-chef and opscode-erchef internal load-balancer error logs.
/var/log/opscode/nginx/nagios.access.log The nagios access logs.
/var/log/opscode/nginx/nagios.error.log The nagios error logs.
/var/log/opscode/nginx/rewrite-port-80.log The rewrite logs for traffic that uses HTTP instead of HTTPS.

To follow the logs for the service:

 $chef-server-ctl tail nginx Chef Client Client.rb3 file might help you. Default value of log_location is STDOUT. You can give /path/to/log_location in place of this. You can locate this client.rb file in C:\chef\client.rb or /etc/chef/client.rb directories. Use the verbose logging that is built into the chef-client: -l LEVEL, –log_level LEVEL The level of logging to be stored in a log file. -L LOGLOCATION, –logfile c The location of the log file. This is recommended when starting any executable as a daemon. Default value: STDOUT. Knife Use the verbose logging that is built into knife: -V, –verbose Set for more verbose outputs. Use -VV for maximum verbosity. chef-solo -l LEVEL, –log_level LEVEL The level of logging to be stored in a log file. The Chef file and folder locations are different on Linux and Windows machines. This article explains the purpose of each file and the location. #### Summary Linux Windows Cookbook location /var/chef/cache/cookbooks C:\chef\cache\cookbooks Chef Client run log /var/log/chef.log First run only C:\chef\chef-client.log Subsequent Chef client runs C:\chef\log\client.log Error log /var/chef/cache/chef-stacktrace.out C:\chef\cache\chef-stacktrace.out Ohai output /var/chef/cache/failed-run-data.json C:\chef\cache\failed-run-data.json Recommended location for custom log files /tmp/cheflog.log C:\Logs\Chef\cheflog.log Chef Client configuration /etc/chef/client.rb C:\chef\client.rb #### When you test your cookbook in Test Kitchen The .kitchen.yml file contains the username to execute the Chef cookbook. It is specified under platforms:, transport:, username: Use that value in place of USER-NAME-FROM-KITCHEN-YML below. Linux Windows Cookbook location /tmp/kitchen/cookbooks /tmp/kitchen/cache/cookbooks C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cookbooks Error log /tmp/kitchen/cache/chef-stacktrace.out C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cache\chef-stacktrace.out Ohai output /tmp/kitchen/cache/failed-run-data.json C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cache\failed-run-data.json Data bags /tmp/kitchen/data_bags C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\data_bags Cookbook location When the Chef recipes are executed, all cookbooks are stored on the node. You can examine the code to make sure your latest changes are reflected on the machine. The log of the Chef client run The output of the Chef cookbook execution is in the chef.log or chef-client.log file On Windows The log of the first Chef Client run and subsequent runs are stored in different log files. After the initial Chef Client run, the rest of the log entries are collected in the second file. Stacktrace Chef saves information on the hard drive when scripts are executed. If there is a failure, the stack trace of the last error is saved in the chef-stacktrace.out file. Ohai output All the information that Ohai collects on the instance, is saved in the failed-run-data.jsonfile, even if there is no error. It is a great resource to get the server specific values. Reference ## Chef notifies and subscribes explained with examples ### Chef notifies and subscribes explained with examples A notification is a property on a resource that listens to other resources in the resource collection and then takes actions based on the notification type (notifies or subscribes). ### Timers A timer specifies the point during the chef-client run at which a notification is run. The following timers are available: :before Specifies that the action on a notified resource should be run before processing the resource block in which the notification is located. :delayed Default. Specifies that a notification should be queued up, and then executed at the very end of the chef-client run. :immediate, :immediately Specifies that a notification should be run immediately, per resource notified. ### Notifies A resource may notify another resource to take action when its state changes. Specify a ‘resource[name]’, the :action that resource should take, and then the :timer for that action. A resource may notify more than one resource; use a notifies statement for each resource to be notified. The syntax for notifies is: notifies :action, ‘resource[name]’, :timer Example The following examples show how to use the notifies notification in a recipe. Delay notifications template ‘/etc/nagios3/configures-nagios.conf’ do # other parameters notifies :run, ‘execute[test-nagios-config]’, :delayed end Notify immediately By default, notifications are :delayed, that is they are queued up as they are triggered, and then executed at the very end of a chef-client run. To run an action immediately, use :immediately: template ‘/etc/nagios3/configures-nagios.conf’ do # other parameters notifies :run, ‘execute[test-nagios-config]’, :immediately end and then the chef-client would immediately run the following: execute ‘test-nagios-config’ do command ‘nagios3 –verify-config’ action :nothing end ### Subscribes A resource may listen to another resource, and then take action if the state of the resource being listened to changes. Specify a ‘resource[name]’, the :action to be taken, and then the :timer for that action. Note that subscribes does not apply the specified action to the resource that it listens to – for example: file ‘/etc/nginx/ssl/example.crt’ do mode ‘0600’ owner ‘root’ end service ‘nginx’ do subscribes :reload, ‘file[/etc/nginx/ssl/example.crt]’, :immediately end In this case the subscribes property reloads the nginx service whenever its certificate file, located under /etc/nginx/ssl/example.crt, is updated. subscribes does not make any changes to the certificate file itself, it merely listens for a change to the file, and executes the :reload action for its resource (in this example nginx) when a change is detected. The syntax for subscribes is: subscribes :action, ‘resource[name]’, :timer Examples The following examples show how to use the subscribes notification in a recipe. Prevent restart and reconfigure if configuration is broken Use the :nothing action (common to all resources) to prevent the test from starting automatically, and then use the subscribes notification to run a configuration test when a change to the template is detected: execute ‘test-nagios-config’ do command ‘nagios3 –verify-config’ action :nothing subscribes :run, ‘template[/etc/nagios3/configures-nagios.conf]’, :immediately end Reference Example ## Chef Internal – How Chef maintain the state of each resources internally? Chef Client A chef-client is an agent that runs locally on every node that is under management by Chef. The chef-client executable can be run as a daemon. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including: • Registering and authenticating the node with the Chef server • Building the node object • Synchronizing cookbooks • Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies • Taking the appropriate and required actions to configure the node • Looking for exceptions and notifications, handling each as required RSA public key-pairs are used to authenticate the chef-client with the Chef server every time a chef-client needs access to data that is stored on the Chef server. This prevents any node from accessing data that it shouldn’t and it ensures that only nodes that are properly registered with the Chef server can be managed. Ohai Ohai is a tool that is used to collect system configuration data, which is provided to the chef-client for use within cookbooks. Ohai is run by the chef-client at the beginning of every Chef run to determine system state. Ohai includes many built-in plugins to detect common configuration details as well as a plugin model for writing custom plugins. The types of attributes Ohai collects include but are not limited to: • Operating System • Network • Memory • Disk • CPU • Kernel • Host names • Fully qualified domain names • Virtualization • Cloud provider metadata Attributes that are collected by Ohai are automatic level attributes, in that these attributes are used by the chef-client to ensure that these attributes remain unchanged after the chef-client is done configuring the node. Ohai collects data for many platforms, including AIX, Darwin, Linux, FreeBSD, OpenBSD, NetBSD, Solaris, and any Microsoft Windows operating systems. ## What is the significance of the default directory under chef cookbook /templates? What is the significance of the default directory under chef cookbook /templates? A cookbook is frequently designed to work across many platforms and is often required to distribute a specific template to a specific platform. This is a New in Chef Client 12.0. A cookbook can be designed to support the distribution of templates across platforms, while ensuring that the correct template ends up on each system. The pattern for template specificity depends on two things: the lookup path and the source. The first pattern that matches is used: /host-$fqdn/$source /$platform-$platform_version/$source
/$platform/$source
/default/$source /$source

Use an array with the source property to define an explicit lookup path. For example:
template ‘/test’ do
source [“#{node.chef_environment}.erb”, ‘default.erb’]
end

The following example emulates the entire file specificity pattern by defining it as an explicit path:
template ‘/test’ do
source %W{
host-#{node[‘fqdn’]}/test.erb
#{node[‘platform’]}-#{node[‘platform_version’]}/test.erb
#{node[‘platform’]}/test.erb
default/test.erb
}
end

A cookbook may have a /templates directory structure like this:
/templates/
windows-6.2
windows-6.1
windows-6.0
windows
default

and a resource that looks something like the following:
template ‘C:\path\to\file\text_file.txt’ do
source ‘text_file.txt’
mode ‘0755’
owner ‘root’
group ‘root’
end

This resource would be matched in the same order as the /templates directory structure. For a node named host-node-desktop that is running Windows 7, the second item would be the matching item and the location:
/templates
windows-6.2/text_file.txt
windows-6.1/text_file.txt
windows-6.0/text_file.txt
windows/text_file.txt

default/text_file.txt