CM Services in Software Environments: A Question of Integration

There is no concensus regarding where CM services should reside in software environment architectures, despite the diversity of approaches that have been explored. For example, CM services have been offered via:

· Tools such as RCS, SCCS, CCC.
· Operating system extensions at the file-system level such as DSEE and NSE.
· Shared data models such as in the CIS specifications [18] and the PCTE PACT [53] environment.

A further complication is the emergence of a robust CASE tool industry, wherein many popular CASE tools provide their own tool-specific repository and CM services. As a result, CM functions are increasingly provided by, and distributed across, several CASE tools in an environment.
We have found it useful to think of integration in terms of a three-level model. This model, illustrated in Figure 5-1, corresponds to the ANSI/SPARC [48] three-schema  pproach used to describe database architectures. A useful intuition is that this correspondence is more than accidental. The bottom level of integration, called “mechanism” integration, corresponds to the ANSI/SPARC physical schema level. Mechanism integration addresses the implementation aspects of software integration, including, but not limited to: software interfaces provided by the environment infrastructure, e.g., operating system or environment framework interfaces;

software interfaces provided by individual tools in the environment; and architectural aspects of the tools, such as process structure (e.g., client/server) and data management structure (derivers, data dictionary, database). In the case of CM, mechanism integration can refer to integration with CM systems such as SCCS, RCS, CCC and DSEE; and CM implementation aspects such as transparent repositories and other operating-systems level CM services.

The middle level of integration, called “services” integration, corresponds to the ANSI/SPARC logical schema level. Services refers to the high-level functions provided by tools, and integration at this level can be regarded as the specification of how services can be related in a coherent fashion. In the case of CM, these services refer to elements of the spectrum of concepts discussed in chapter 3, e.g., workspaces and transactions, and services integration constitutes a kind of unified model of CM services.

The top level of integration, called “process” integration, corresponds to the ANSI/SPARC external schema (also called “end-user”) level. Process integration can be regarded as a kind of process specification for how software will be developed; this specification can define a view of the process from many perspectives, spanning individual roles through larger organizational pespectives. In the case of CM, process integration refers to policies and procedures for carrying out CM activities.

Integration occurs within each of these levels of integration; thus, mechanisms are inte- 34 ATR grated with mechanisms, services with services, and process elements with process elements. There are also relationships that span the levels. The relationship between the mechanism level and the services level is an implementation relationship: a CM concept in  he services layer may be implemented by different tools in the mechanism level, and conversely, a single mechanism may implement more than one CM concept. The relationship between the services level and the process level is a process adaptation relationship: different CM services may be combined, and tuned, to support different process requirements.

This three-level model provides a working context for understanding integration. For the moment, however, existing integration technology does not match exactly this somewhat idealized model of integration. For example, many services provided by CASE tools (including CM) embed process constraints that should logically be separate, i.e., reside in the process level. Similarly, tool services are often closely coupled to particular implementation techniques.

The level of adaptability required of integrating CM—both in terms of adaptability for projectspecific requirements as well as adaptability to multiple underlying CM
implementations—pushes the limits of available environment integration techniques. The following sections describe the current state of integration technology and its limitations. The next chapter discusses how future generation integration technology can address these shortcomings.

Reference: 
The State of Automated Configuration Management.
A. Brown, S. Dart, P. Feiler, K. Wallnau

The post General SCM Interview Questions appeared first on scmGalaxy.

• Identification: an identification scheme reflects the structure of the product, identifies components and their types, making them unique and accessible in some form.
• Control: controlling the release of a product and changes to it throughout the lifecycle by having controls in place that ensure consistent software via the creation of a baseline product.
• Status Accounting: recording and reporting the status of components and change requests, and gathering vital statistics about components in the product.
• Audit and review: validating the completeness of a product and maintaining consistency among the components by ensuring that the product is a welldefined collection of components.

The definition includes terminology such as configuration item, baseline, release and version. When analyzing CM systems—automated tools that provide CM—it becomes evident that these incorporate functionality of varying degrees to support the above definition. Some CM systems provide functionality that goes beyond the above definition though. This is due (among other reasons) to the recognition of different user roles, disparate operating environments such as heterogeneous platforms, and programming-in-the-large support such as enabling teams of software programmers to work on large projects synergistically. To capture this extra functionality, it is necessary to broaden the definition of CM to include:

• Manufacture: managing the construction and building of the product in an effective and optimal manner.
• Process management: ensuring the carrying out of the organization’s procedures, policies and lifecycle model.
• Team work: controlling the work and interactions between multiple users on a product.

In summary, the capabilities provided by existing CM systems encompass identification, control, status accounting, audit and review, manufacture, process management and team work.

The post Definition of Configuration Management appeared first on scmGalaxy.

A simple, typical, CM user scenario of an organization is described in order to present the various roles and the subsequent views of CM support. The scenario involves various people with different responsibilities: a project manager who is in charge of a software group, a configuration manager who is in charge of the CM procedures and policies, the programmers who are responsible for developing and maintaining the software product, the tester who validates the correctness of the product, the quality assurance (QA) manager who ensures the high quality of the product, and the customer who uses the product. Each role comes with its own goals and tasks. For the project manager, major goals are to ensure that the product is developed within a certain time frame and meets the customer’s  requirements. Hence, the manager monitors the progress of development and recognizes and reacts to problems. This is done by generating and analyzing reports about the status of the software system and by performing reviews on the system.

The goals of the configuration manager are to ensure that procedures and policies for creating,
changing, and testing of code are defined and followed, as well as to make information about the project accessible. To implement techniques for maintaining control over code changes, this manager introduces mechanisms for making official requests for changes, for evaluating changes (via a Change Control Board (CCB) that is responsible for approving changes to the software system), and for authorizing changes. The manager creates and disseminates task lists for the programmers and basically creates the project context. Also, the manager collects statistics about components in the software system, such as information determining which components in the system are problematic.

For the programmers, the goal is to work effectively in creating the product. This means programmers do not unnecessarily interfere with each other in the creation and testing of code and in the production of supporting documents. But, at the same time, they communicate and coordinate efficiently. They use tools that help build a consistent software product, and they communicate and coordinate by notifying one another about tasks required and tasks completed. Changes are propagated between each other’s work by merging those changes and resolving any conflicts. A history is kept of the evolution of all components in the product along with a log of what actually changed and reasons for those
changes. The programmers have their own work area for creating, changing, testing, and integrating code. At a certain point, the code is made into a baseline from which further development continues and from which parallel development for variants of other target machines emerges.
The tester’s goal is to make sure every component of the product is tested and found satis-factory. This involves testing a particular version of the product and keeping a record of which tests apply to which version of the product along with the results of the tests. Any errors are reported back to the appropriate people, tracked and fixes are put through regression testing.

The Quality Assurance (QA) manager’s goal is to ensure the high quality of the product. This means that certain procedures and policies must be fulfilled with the appropriate approval. Bugs must be fixed and fixes propagated to the appropriate variants of the product with the correct amount of testing applied to each variant. Customer complaints about the product must be followed up.

The customers use the product — most likely, different customers use different versions and variants of it. Customers follow certain procedures for requesting changes and for indicating bugs and improvements for the product.

As a result of the various roles, an organization ends up with several views of CM. These views range from corporate CM (CCM) and project CM (PCM) to developer CM (DCM) and application CM (ACM). The former two views treat CM as a management and control discipline emphasizing the process aspect of CM, while the latter two views treat CM as a developer support function emphasizing tool support for CM. CCM focuses on maintaining CM information for a product, whose data is the basis for strategic impact analysis and process improvement. PCM focuses on managing change to a product through formal control processes such as change authorization via CCBs and configuration managers guarding CM repositories. DCM focuses on supporting developers (programmers) performing actual changes, maintaining history of actual changes, and providing stability and consistency through system build and team support. ACM focuses on configuration of applications as they are deployed, addressing issues of installation management, as well as dynamic configuration and reconfiguration. Ideally, a CM system suited to the views should support all these goals, roles and tasks.

Reference: 
The State of Automated Configuration Management
A. Brown, S. Dart, P. Feiler, K. Wallnau

The post Roles of CM Users and Views of CM appeared first on scmGalaxy.

The post Configuration Management appeared first on scmGalaxy.

Naturally, vendors rush into this environment, each touting the unique benefits of its solution set and the urgency to move forward immediately.  A key area targeted for IT automation is the area of ‘configuration management.’  As it relates to automating day to day IT functions, configuration management can mean many different things: patch management, server and network management or others.

The post 5 Keys to Automating Configuration Management for Application Infrastructure appeared first on scmGalaxy.

Source – learn.chef.io

What is a resource?
Answer- A resource represents a piece of infrastructure and its desired state, such as a package that should be installed, a service that should be running, or a file that should be generated.

Question: What is a recipe?
Answer- A recipe is a collection of resources that describes a particular configuration or policy. A recipe describes everything that is required to configure part of a system. Recipes do things such as:

install and configure software components.
manage files.
deploy applications.
execute other recipes.

Question: What happens when you don’t specify a resource’s action?
Answer- When you don’t specify a resource’s action, Chef applies the default action.

Question: Are these two recipes the same?

package 'httpd'

service 'httpd' do    action [:enable, :start]  end

&&

service 'httpd' do    action [:enable, :start]    end

package 'httpd'

Answer-
No, they are not. Remember that Chef applies resources in the order they appear. So the first recipe ensures that the httpd package is installed and then configures the service. The second recipe configures the service and then ensures the package is installed.

Question: The second recipe may not work as you’d expect because the service resource will fail if the package is not yet installed.

Are these two recipes the same?

package 'httpd'

service 'httpd' do    action [:enable, :start]    end

package 'httpd'

service 'httpd' do    action [:start, :enable]    end

Answer-
No, they are not. Although both recipes ensure that the httpd package is installed before configuring its service, the first recipe enables the service when the system boots and then starts it. The second recipe starts the service and then enables it to start on reboot.

Are these two recipes the same?

file ‘/etc/motd’ do
owner ‘root’
group ‘root’
mode ‘0755’
action :create
end

file ‘/etc/motd’ do
action :create
mode ‘0755’
group ‘root’
owner ‘root’
end

Answer-
Yes, they are! Order matters with a lot of things in Chef, but you can order resource attributes any way you want.

Question –
Write a service resource that stops and then disables the httpd service from starting when the system boots.

Answer –
service ‘httpd’ do
action [:stop, :disable]
end

How does a cookbook differ from a recipe?
A recipe is a collection of resources, and typically configures a software package or some piece of infrastructure. A cookbook groups together recipes and other information in a way that is more manageable than having just recipes alone.

For example, in this lesson you used a template resource to manage your HTML home page from an external file. The recipe stated the configuration policy for your web site, and the template file contained the data. You used a cookbook to package both parts up into a single unit that you can later deploy.

How does chef-apply differ from chef-client?

chef-apply applies a single recipe; chef-client applies a cookbook.

For learning purposes, we had you start off with chef-apply because it helps you understand the basics quickly. In practice, chef-apply is useful when you want to quickly test something out. But for production purposes, you typically run chef-client to apply one or more cookbooks.

You’ll learn in the next module how to run chef-client remotely from your workstation.

What’s the run-list?

The run-list lets you specify which recipes to run, and the order in which to run them. The run-list is important for when you have multiple cookbooks, and the order in which they run matters.

What are the two ways to set up a Chef server?

Install an instance on your own infrastructure.
Use hosted Chef.

What’s the role of the Starter Kit?
The Starter Kit provides certificates and other files that enable you to securely communicate with the Chef server.

Where can you get reusable cookbooks that are written and maintained by the Chef community?
Chef Supermarket, https://supermarket.chef.io.

What’s the command that enables you to interact with the Chef server?
knife

What is a node?
A node represents a server and is typically a virtual machine, container instance, or physical server – basically any compute resource in your infrastructure that’s managed by Chef.

What information do you need to in order to bootstrap?
You need:

your node’s host name or public IP address.
a user name and password you can log on to your node with.
Alternatively, you can use key-based authentication instead of providing a user name and password.

What happens during the bootstrap process?
During the bootstrap process, the node downloads and installs chef-client, registers itself with the Chef server, and does an initial checkin. During this checkin, the node applies any cookbooks that are part of its run-list.

Which of the following lets you verify that your node has successfully bootstrapped?

The Chef management console.
knife node list
knife node show
You can use all three of these methods.

What is the command you use to upload a cookbook to the Chef server?
knife cookbook upload

How do you apply an updated cookbook to your node?
We mentioned two ways.

Run knife ssh from your workstation.
SSH directly into your server and run chef-client.
You can also run chef-client as a daemon, or service, to check in with the Chef server on a regular interval, say every 15 or 30 minutes.

Update your Apache cookbook to display your node’s host name, platform, total installed memory, and number of CPUs in addition to its FQDN on the home page.

Update index.html.erb like this.
<html>
<body>
<h1>hello from <%= node[‘fqdn’] %></h1>

<pre>
<%= node[‘hostname’] %>
<%= node[‘platform’] %> – <%= node[‘platform_version’] %>
<%= node[‘memory’][‘total’] %> RAM
<%= node[‘cpu’][‘total’] %> CPUs
</pre>
</body>
</html>

Then upload your cookbook and run it on your node.

What would you set your cookbook’s version to once it’s ready to use in production?

According to Semantic Versioning, you should set your cookbook’s version number to 1.0.0 at the point it’s ready to use in production.

What is the latest version of the haproxy community cookbook?

To know the latest version of any cookbook on Chef Supermarket, browse to its page and view the latest version from the version selection box.

Or, get the info from the knife cookbook site command, like this.

knife cookbook site show haproxy | grep latest_version
latest_version: http://cookbooks.opscode.com/api/v1/cookbooks/haproxy/versions/1.6.6

Create a second node and apply the awesome_customers cookbook to it. How long does it take?

You already accomplished the majority of the tasks that you need. You wrote the awesome_customers cookbook, uploaded it and its dependent cookbooks to the Chef server, applied the awesome_customers cookbook to your node, and verified that everything’s working.

All you need to do now is:

Bring up a second Red Hat Enterprise Linux or CentOS node.
Copy your secret key file to your second node.
Bootstrap your node the same way as before. Because you include the awesome_customers cookbook in your run-list, your node will apply that cookbook during the bootstrap process.
The result is a second node that’s configured identically to the first one. The process should take far less time because you already did most of the work.

Now when you fix an issue or add a new feature, you’ll be able to deploy and verify your update much more quickly!

What’s the value of local development using Test Kitchen?

Local development with Test Kitchen:

enables you to use a variety of virtualization providers that create virtual machine or container instances locally on your workstation or in the cloud.
enables you to run your cookbooks on servers that resemble those that you use in production.
speeds up the development cycle by automatically provisioning and tearing down temporary instances, resolving cookbook dependencies, and applying your cookbooks to your instances.

What is VirtualBox? What is Vagrant?

VirtualBox is the software that manages your virtual machine instances.

Vagrant helps Test Kitchen communicate with VirtualBox and configures things like available memory and network settings.

Verify that your motd cookbook runs on both CentOS 6.6 and CentOS 6.5.

Your motd cookbook is already configured to work on CentOS 6.6 as well as CentOS 6.5, so you don’t need to modify it.

To run it on CentOS 6.5, add an entry to the platforms section of your .kitchen.yml file like this.

---       driver:       name: vagrant

provisioner:       name: chef_zero

platforms:       - name: centos-6.6       driver:       box: opscode-centos-6.6       box_url: http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.6_chef-provisionerless.box       - name: centos-6.5       driver:       box: opscode-centos-6.5       box_url: http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.5_chef-provisionerless.box

suites:       - name: default       run_list:       - recipe[motd::default]       attributes:

In many cases, Test Kitchen can infer the box and box_url parameters, which specify the name and location of the base image, or box. We specify them here to show you how to use them.

Run kitchen list to see the matrix of test instances that are available. Here, we have two platforms – CentOS 6.5 and CentOS 6.6 – multiplied by one suite – default.

$kitchen list

Instance           Driver   Provisioner  Verifier  Transport  Last Action default-centos-66  Vagrant  ChefZero     Busser    Ssh        <Not Created> default-centos-65  Vagrant  ChefZero     Busser    Ssh        <Not Created>

Run kitchen converge to create the instances and apply the motd cookbook.

$kitchen converge    -----> Starting Kitchen (v1.4.0)    -----> Creating <default-centos-66>...    Bringing machine 'default' up with 'virtualbox' provider...    [...]    Running handlers:    Running handlers complete    Chef Client finished, 1/1 resources updated in 10.372334751 seconds    Finished converging <default-centos-66> (3m52.59s).    -----> Creating <default-centos-65>...    Bringing machine 'default' up with 'virtualbox' provider...    [...]    Running handlers:    Running handlers complete    Chef Client finished, 1/1 resources updated in 5.32753132 seconds    Finished converging <default-centos-65> (10m12.63s).  -----> Kitchen is finished. (19m47.71s)

Now to confirm that everything’s working, run kitchen login. But this time, you need to provide the instance name so that Test Kitchen knows which instance to connect to.

$kitchen login default-centos-66       Last login: Wed May 13 20:15:00 2015 from 10.0.2.2              hostname:  default-centos-66       fqdn:      default-centos-66       memory:    469392kB       cpu count: 1       [vagrant@default-centos-66 ~]$ logout       Connection to 127.0.0.1 closed.$kitchen login default-centos-65       Last login: Wed May 13 20:28:18 2015 from 10.0.2.2              hostname:  default-centos-65       fqdn:      default-centos-65       memory:    469452kB       cpu count: 1       [vagrant@default-centos-65 ~]$ logout       Connection to 127.0.0.1 closed.

The post Chef configuration management interview questions and answers | Chef Interview Q&A appeared first on scmGalaxy.

Source – learn.chef.io

What is a resource?
Answer- A resource represents a piece of infrastructure and its desired state, such as a package that should be installed, a service that should be running, or a file that should be generated.

Question: What is a recipe?
Answer- A recipe is a collection of resources that describes a particular configuration or policy. A recipe describes everything that is required to configure part of a system. Recipes do things such as:

install and configure software components.
manage files.
deploy applications.
execute other recipes.

Question: What happens when you don’t specify a resource’s action?
Answer- When you don’t specify a resource’s action, Chef applies the default action.

Question: Are these two recipes the same?

package 'httpd'

service 'httpd' do    action [:enable, :start]  end

&&

service 'httpd' do    action [:enable, :start]    end

package 'httpd'

Answer-
No, they are not. Remember that Chef applies resources in the order they appear. So the first recipe ensures that the httpd package is installed and then configures the service. The second recipe configures the service and then ensures the package is installed.

Question: The second recipe may not work as you’d expect because the service resource will fail if the package is not yet installed.

Are these two recipes the same?

package 'httpd'

service 'httpd' do    action [:enable, :start]    end

package 'httpd'

service 'httpd' do    action [:start, :enable]    end

Answer-
No, they are not. Although both recipes ensure that the httpd package is installed before configuring its service, the first recipe enables the service when the system boots and then starts it. The second recipe starts the service and then enables it to start on reboot.

Are these two recipes the same?

file ‘/etc/motd’ do
owner ‘root’
group ‘root’
mode ‘0755’
action :create
end

file ‘/etc/motd’ do
action :create
mode ‘0755’
group ‘root’
owner ‘root’
end

Answer-
Yes, they are! Order matters with a lot of things in Chef, but you can order resource attributes any way you want.

Question –
Write a service resource that stops and then disables the httpd service from starting when the system boots.

Answer –
service ‘httpd’ do
action [:stop, :disable]
end

How does a cookbook differ from a recipe?
A recipe is a collection of resources, and typically configures a software package or some piece of infrastructure. A cookbook groups together recipes and other information in a way that is more manageable than having just recipes alone.

For example, in this lesson you used a template resource to manage your HTML home page from an external file. The recipe stated the configuration policy for your web site, and the template file contained the data. You used a cookbook to package both parts up into a single unit that you can later deploy.

How does chef-apply differ from chef-client?

chef-apply applies a single recipe; chef-client applies a cookbook.

For learning purposes, we had you start off with chef-apply because it helps you understand the basics quickly. In practice, chef-apply is useful when you want to quickly test something out. But for production purposes, you typically run chef-client to apply one or more cookbooks.

You’ll learn in the next module how to run chef-client remotely from your workstation.

What’s the run-list?

The run-list lets you specify which recipes to run, and the order in which to run them. The run-list is important for when you have multiple cookbooks, and the order in which they run matters.

What are the two ways to set up a Chef server?

Install an instance on your own infrastructure.
Use hosted Chef.

What’s the role of the Starter Kit?
The Starter Kit provides certificates and other files that enable you to securely communicate with the Chef server.

Where can you get reusable cookbooks that are written and maintained by the Chef community?
Chef Supermarket, https://supermarket.chef.io.

What’s the command that enables you to interact with the Chef server?
knife

What is a node?
A node represents a server and is typically a virtual machine, container instance, or physical server – basically any compute resource in your infrastructure that’s managed by Chef.

What information do you need to in order to bootstrap?
You need:

your node’s host name or public IP address.
a user name and password you can log on to your node with.
Alternatively, you can use key-based authentication instead of providing a user name and password.

What happens during the bootstrap process?
During the bootstrap process, the node downloads and installs chef-client, registers itself with the Chef server, and does an initial checkin. During this checkin, the node applies any cookbooks that are part of its run-list.

Which of the following lets you verify that your node has successfully bootstrapped?

The Chef management console.
knife node list
knife node show
You can use all three of these methods.

What is the command you use to upload a cookbook to the Chef server?
knife cookbook upload

How do you apply an updated cookbook to your node?
We mentioned two ways.

Run knife ssh from your workstation.
SSH directly into your server and run chef-client.
You can also run chef-client as a daemon, or service, to check in with the Chef server on a regular interval, say every 15 or 30 minutes.

Update your Apache cookbook to display your node’s host name, platform, total installed memory, and number of CPUs in addition to its FQDN on the home page.

Update index.html.erb like this.
<html>
<body>
<h1>hello from <%= node[‘fqdn’] %></h1>

<pre>
<%= node[‘hostname’] %>
<%= node[‘platform’] %> – <%= node[‘platform_version’] %>
<%= node[‘memory’][‘total’] %> RAM
<%= node[‘cpu’][‘total’] %> CPUs
</pre>
</body>
</html>

Then upload your cookbook and run it on your node.

What would you set your cookbook’s version to once it’s ready to use in production?

According to Semantic Versioning, you should set your cookbook’s version number to 1.0.0 at the point it’s ready to use in production.

What is the latest version of the haproxy community cookbook?

To know the latest version of any cookbook on Chef Supermarket, browse to its page and view the latest version from the version selection box.

Or, get the info from the knife cookbook site command, like this.

knife cookbook site show haproxy | grep latest_version
latest_version: http://cookbooks.opscode.com/api/v1/cookbooks/haproxy/versions/1.6.6

Create a second node and apply the awesome_customers cookbook to it. How long does it take?

You already accomplished the majority of the tasks that you need. You wrote the awesome_customers cookbook, uploaded it and its dependent cookbooks to the Chef server, applied the awesome_customers cookbook to your node, and verified that everything’s working.

All you need to do now is:

Bring up a second Red Hat Enterprise Linux or CentOS node.
Copy your secret key file to your second node.
Bootstrap your node the same way as before. Because you include the awesome_customers cookbook in your run-list, your node will apply that cookbook during the bootstrap process.
The result is a second node that’s configured identically to the first one. The process should take far less time because you already did most of the work.

Now when you fix an issue or add a new feature, you’ll be able to deploy and verify your update much more quickly!

What’s the value of local development using Test Kitchen?

Local development with Test Kitchen:

enables you to use a variety of virtualization providers that create virtual machine or container instances locally on your workstation or in the cloud.
enables you to run your cookbooks on servers that resemble those that you use in production.
speeds up the development cycle by automatically provisioning and tearing down temporary instances, resolving cookbook dependencies, and applying your cookbooks to your instances.

What is VirtualBox? What is Vagrant?

VirtualBox is the software that manages your virtual machine instances.

Vagrant helps Test Kitchen communicate with VirtualBox and configures things like available memory and network settings.

Verify that your motd cookbook runs on both CentOS 6.6 and CentOS 6.5.

Your motd cookbook is already configured to work on CentOS 6.6 as well as CentOS 6.5, so you don’t need to modify it.

To run it on CentOS 6.5, add an entry to the platforms section of your .kitchen.yml file like this.

---       driver:       name: vagrant

provisioner:       name: chef_zero

platforms:       - name: centos-6.6       driver:       box: opscode-centos-6.6       box_url: http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.6_chef-provisionerless.box       - name: centos-6.5       driver:       box: opscode-centos-6.5       box_url: http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.5_chef-provisionerless.box

suites:       - name: default       run_list:       - recipe[motd::default]       attributes:

In many cases, Test Kitchen can infer the box and box_url parameters, which specify the name and location of the base image, or box. We specify them here to show you how to use them.

Run kitchen list to see the matrix of test instances that are available. Here, we have two platforms – CentOS 6.5 and CentOS 6.6 – multiplied by one suite – default.

$kitchen list

Instance           Driver   Provisioner  Verifier  Transport  Last Action default-centos-66  Vagrant  ChefZero     Busser    Ssh        <Not Created> default-centos-65  Vagrant  ChefZero     Busser    Ssh        <Not Created>

Run kitchen converge to create the instances and apply the motd cookbook.

$kitchen converge    -----> Starting Kitchen (v1.4.0)    -----> Creating <default-centos-66>...    Bringing machine 'default' up with 'virtualbox' provider...    [...]    Running handlers:    Running handlers complete    Chef Client finished, 1/1 resources updated in 10.372334751 seconds    Finished converging <default-centos-66> (3m52.59s).    -----> Creating <default-centos-65>...    Bringing machine 'default' up with 'virtualbox' provider...    [...]    Running handlers:    Running handlers complete    Chef Client finished, 1/1 resources updated in 5.32753132 seconds    Finished converging <default-centos-65> (10m12.63s).  -----> Kitchen is finished. (19m47.71s)

Now to confirm that everything’s working, run kitchen login. But this time, you need to provide the instance name so that Test Kitchen knows which instance to connect to.

$kitchen login default-centos-66       Last login: Wed May 13 20:15:00 2015 from 10.0.2.2              hostname:  default-centos-66       fqdn:      default-centos-66       memory:    469392kB       cpu count: 1       [vagrant@default-centos-66 ~]$ logout       Connection to 127.0.0.1 closed.$kitchen login default-centos-65       Last login: Wed May 13 20:28:18 2015 from 10.0.2.2              hostname:  default-centos-65       fqdn:      default-centos-65       memory:    469452kB       cpu count: 1       [vagrant@default-centos-65 ~]$ logout       Connection to 127.0.0.1 closed.

The post Top 25 Chef configuration management interview questions and answers appeared first on scmGalaxy.

Top 5 Configuration Management Tools

Puppet

In computing, Puppet is an open-source configuration management utility. It runs on many Unix-like systems as well as on Microsoft Windows, and includes its own declarative language to describe system configuration.

Puppet is produced by Puppet Labs, founded by Luke Kanies in 2005. It is written in Ruby and released as free software under the GNU General Public License (GPL) until version 2.7.0 and the Apache License 2.0 after that.

What started out as a popular DevOps tool has quickly become a movement. Written in Ruby, like Chef, Puppet also comes in both an open source and enterprise version. However, where Chef has a healthy offering of features across both open source and enterprise versions, Puppet has placed the majority of its feature set into enterprise status. Features that the open source version comes with include provisioning (Amazon EC2, Google Compute Engine), configuration management (operating systems and applications) plus 2,000+ pre-built configurations on Puppet Forge. Considerably more features are available for the enterprise version, including the open source features plus graphical user interface, event inspector (visualize infrastructure changes), supported modules, and provisioning (VMware VMs). Configuration management (discovery, user accounts), orchestration, task automation, role-based access control (with external authentication support) are also included. Puppet enterprise has a unified cross-platform installer of all components and support.

Chef

Offered as both an open source and enterprise product, Chef is a powerful tool for full IT infrastructure configuration management. With open source Chef at the heart of both offerings, shared features include a flexible and scalable automation platform, access to 800+ reusable cookbooks and integration with leading cloud providers. Chef also offers enterprise platform support, including Windows and Solaris, and allows you to create, bootstrap and manage OpenStack clouds. It has easy installation with ‘one-click’ Omnibus Installer, automatic system discovery with Ohai, text-based search capabilities and multiple environment support. Other notable features inclyde the “Knife” command line interface, “Dry Run” mode for testing potential changes, and the ability to manage 10,000+ nodes on a single Chef server. Features only available in the enterprise version of Chef include availablility as a hosted service, enhanced management console, centralized activity and resource reporting, as well as “Push” command and control client runs. Multi-tenancy, role-based access control (RBAC), high availability installation support and verification, along with centralized authentication using LDAP or Active Directory are included with Chef enterprise.

Ansible

Ansible is a model-driven configuration management tool that leverages SSH to improve security and simplify management. In addition to configuration management, it is capable of automating app deployment (even multi-tier deployment), workflow orchestration and cloud provisioning, hence the company likes the tool to be categorized as an “orchestration engine.” Ansible is built on five design principles including ease of use (doesn’t require writing scripts or custom code), low learning curve (both for sysadmins and developers), comprehensive automation (allowing you to automate almost anything in your environment), efficiency (since it runs on OpenSSH it doesn’t rely on memory or CPU resources), and security (it is inherently more secure because it doesn’t require an agent, additional ports or root level daemons). As many other open source projects, Ansible has a paid product that comes in the form of a web UI called Ansible Tower.

CFEngine

One of the earliest full-featured configuration management systems out there, CFEngine has gone through several iterations and maintained relevance as OS have gone from the local data center to the cloud. At the heart of the infrastructure automation framework, CFEngine is also a modeling and monitoring compliance engine, capable of sitting on a small footprint. As recommended by CFEngine, steps toward identifying an initial desired state include: 1) model the desired state of your environment; 2) simulate configuration changes before committing them; 3) confirm the desired state and set for automatic self-healing; 4) collect reports on the differences between actual and desired states. CFEngine has a library of reusable data-driven models that will help users model their desired states. These infrastructure patterns are designed to be reusable across the Enterprise.

Salt

As part of a larger, enterprise ready application, the configuration management piece of Salt is as robust and feature-full as would be expected. Built upon the remote execution core, execution of the system occurs on “minions” which receives commands from the central Salt master and replies with the results of said commands. Salt support simultaneous configuration of tens of thousands of hosts. Based upon host “states”, no programming is required to write the configuration files, which are small and easy to understand, that help identify the state of each host. Additionally, for those who do program, or admins who want to have more control and familiarity with their configuration files, any language can be used to render the configurations.

The post Top 5 Configuration Management Tools appeared first on scmGalaxy.

SCM Process and smartBuild

The post SCM Process and smartBuild Overview, What is SCM Process and smartBuild? appeared first on scmGalaxy.

The practices and procedures for administering source code, producing software development builds, controlling change, and managing software configurations. Specifically, Software Configuration Management ensures the integrity, reliability and reproducibility of developing software products from conception to release.

I should say CM in Pakistan has been catching up slow but very steadily. There are not many senior people in Pakistan who have dedicated their careers in Configuration Management. Many people moved on to various management roles out of CM after some years of experience. This is due to lack of management opportunities in this area. But, times have changed for good and are getting better. With the growing maturity of CM across the industry, organizations are giving utmost importance to this area and so are hiring people at very senior levels.

I have around 3 years of experience while Interacting CM Professionals internationally and national level; found very strong maturity and awareness. With my experience, I would say Pakistan IT firms still need a broader thinking from CM perspective. In many organizations, people think that CM is just about making builds and controlling source code for projects. We need to educate people about the real importance of Software Configuration Management in any organization. There is a need of clear understanding of various aspects of Configuration Management like Change Management, Incident/Problem Management, Build & release management, Code control/version control , Automated Build management , Parallel development and Continuous integration , Introduction to new tools for software and hardware configurations etc…

CM is much beyond a tool administration or builds management. It does comprise entire life cycle of a project as well as the POST PRODUCTION analysis to improve the process for upcoming projects. It’s a continuous process which not only gives any IT organization a process to boost confidence in delivering quality product with stable code line.

NetSol is vigilant and moving superbly toward innovation and automation through R&D in the field of Configuration Management. Being the CMMI 5 organization; we have implemented excellent Release Management Process that beautifully interacting and entertaining CM Services throughout the organization. We are in process of experimenting Modern tools and discovering best practices for making our process incredibly easy and technically strong. NetSol’s Department of Configuration Management really welcomes newly born organizations and specially those (existing) who want to setup/improve their Configuration Management process and looking for technical improvement mentor; please come up and Consult NetSol with trust.

The post Software Configuration Management in Pakistan | SCM Practices in Pakistan appeared first on scmGalaxy.

My main experience is with ClearCase. I have read various descriptive comparisons between ClearCase and other available Software Management Version tools. Which tool has your company implemented? What were the key features helped you select this tool over another one?

Configuration Management Tools Discussion

The post Configuration Management Tools Discussion appeared first on scmGalaxy.

Power Point PPT: Introduction To Software Configuration Management

The post Power Point PPT: Introduction To Software Configuration Management appeared first on scmGalaxy.

Power Point PPT: Software Configuration Management And CVS

The post Power Point PPT: Software Configuration Management And CVS appeared first on scmGalaxy.

Power Point PPT: Why Software Configuration Management

The post Power Point PPT: Why Software Configuration Management – Prsesentation appeared first on scmGalaxy.

Why Worry About Versioning?

Having a good version scheme for your software is important for several reasons. The following are the top five things a version scheme allows you to do (in random order):

Track your product binaries to the original source files.

Re-create a past build by having meaningful labels in your source tree.

Avoid “DLL hell” —multiple versions of the same file (library in this case) on a machine.

Help your setup program handle upgrades and service packs.

Provide your product support and Q/A teams with an easy way to identify the bits they are working with.

So, how do you keep track of files in a product and link them back to the owner? How can you tell that you are testing or using the latest version of a released file? What about the rest of the reasons in the list? This chapter describes my recommendations for the most effective and easiest way to set up and apply versioning to your software. Many different schemes are available, and you should feel free to create your own versioning method.

Ultimately, like most of the other topics in this book, the responsibility to apply versioning to the files in a build tends to fall into the hands of the build team. That might be because it is usually the build team that has to deal with the headaches that come from having a poor versioning scheme. Therefore, it is in their best interest to publish, promote, and enforce a good versioning scheme. If the build team does not own this, then someone who does not understand the full implications of versioning will make the rules. Needless to say, this would not be desirable for anybody involved with the product.

File Versioning

Every file in a product should have a version number; a four-part number separated by periods such as the one that follows seems to be the established best practice. There are many variations of what each part represents. I will explain what I think is the best way of defining these parts.

…

Major version— The component owner usually assigns this number. It should be the internal version of the product. It rarely changes during the development cycle of a product release.

Minor version— The component owner usually assigns this number. It is normally used when an incremental release of the product is planned instead of a full feature upgrade. It rarely changes during the development cycle of a product release.

Build number— The build team usually assigns this number based on the build that the file was generated with. It changes with every build of the code.

Revision— The build team usually assigns this number. It can have several meanings: bug number, build number of an older file being replaced, or service pack number. It rarely changes. This number is used mostly when servicing the file for an external release.

Build Number

Each build that is released out of the Central Build Lab should have a unique version stamp (also known as the build number). This number should be incremented just before a build is started. Don’t use a date for a build number or mix dates into a version number simply because there are so many date formats out there that it can be difficult to standardize on one. Also, if you have more than one build on a given date, the naming can get tricky. Stick with an n=n+1 build number. For example, if you release build 100 in the morning and rebuild and release your code in the afternoon, the afternoon build should be build 101. If you were to use a date for the build number, say 010105, what would your afternoon build number be? 010105.1? This can get rather confusing.

It is also a good idea to touch all the files just prior to releasing the build so that you get a current date/time stamp on the files. By “touching” the files, I simply mean using a tool (touch.exe) to modify the date/time stamp on a file. There are several free tools available for you to download, or you can write one yourself. Touching the files helps in the tracking process of the files and keeps all of the dates and times consistent in a build release. It also eliminates the need to include the current date in a build number. You already have the date by just looking at the file properties.

In addition, try to avoid tools that inject version numbers into binaries (as a post-build step). Although the tools might seem reliable, they introduce an instability factor into your released binaries by hacking hexadecimal code. Most Q/A teams become distressed if this happens to the binaries they are testing—and justifiably so. The build number should be built into the binary or document files.

Source Code Control Trees

All the source code control (SCC) software that I have seen has some kind of labeling function to track either checked-in binaries or source code. (Remember that I am not for checking in binaries, but I mention this for the groups or teams that do. This type of versioning (or more appropriately named labeling) is typically used to track a group of sources that correspond to a product release. Most of the time, they combine labeling of the sources with the branching of the source code lines.

Should There Be Other Fields in the File Version Number?

I am of the opinion that no, there shouldn’t be other fields in the file version number. Let’s look at some other fields that might seem like they should be included but really don’t need to be:

• Virtual Build Lab (VBL) or offsite development group number— You can use this number to track a check-in back to a specific site or lab that the code was worked on. If you have a golden tree or mainline setup, and all your VBLs or offsite trees have to reverse integrate into the golden tree, this extra field would be overkill. That’s because you can trace the owner through the golden tree check-in. Having a field in which you would have to look up the VBL or offsite number would take just as long.

  The reality is that when you check a version number, you won’t care where the file came from. You’ll only care if it is a unique enough number to accurately trace the file to the source that created it. Most likely, you’ll already know the version number you’re looking for and you’ll just need to confirm that you’re using it.

• Component—If you break your whole project into separate components, should each component have its own identification number that would be included in the version string? No, similarly to the reasons in the previous bullet, you can track this information by the name of the binary or other information when checking the other properties of the file. This information would probably only come into play if you were filing a bug and you had other resources available to determine which component this file belonged to.
• Service Pack Build Number— If you’re doing daily builds of a service pack release, you should use the earlier example; increment the build number and keep the revision number at the current in-place file build number. This seems like a good argument for a fifth field, but it isn’t if the revision field is used properly.

DLL or Executable Versions for .NET (Assembly Versions)

This section applies to you only if you are programming in .NET.

When Microsoft introduced .NET, one of its goals was to get rid of DLL hell and all the extra setup steps I talk about later in this chapter. In reviewing where .NET is today, it looks like Microsoft has resolved the sideby-side DLL hell problem, but now the problem is “assembly version hell.” Without going into too much detail about the .NET infrastructure, let’s touch on the difference between file versioning and assembly versioning.

Assembly versions are meant for binding purposes only; they are not meant for keeping track of different daily versions. Use the file version for that instead. It is recommended that you keep the assembly version the same from build to build and change it only after each external release. For more details on how .NET works with these versions, refer to Jeffrey Richter’s .NET book. Don’t link the two versions, but make sure each assembly has both an assembly version and a file version when you right-click. You can use the same format described earlier for file versioning for the assembly version.

How Versioning Affects Setup

Have you ever met someone who reformats his machine every six months because “it just crashes less if I do” or it “performs better?” It might sound draconian, but the current state of component versioning and setup makes starting from scratch a likely solution to these performance issues, which are further complicated by spyware.

Most of the problems occur when various pieces of software end up installing components (DLLs and COM components) that are not quite compatible with each other or with the full set of installed products. Just one incorrect or incorrectly installed DLL can make a program flaky or prevent it from starting up. In fact, DLL and component installation is so important that it is a major part of the Windows logo requirement.

If you are involved in your product’s setup, or if you are involved in making decisions about how to update your components (produce new versions), you can do some specific things to minimize DLL hell and get the correct version of your file on the machine.

Installing components correctly is a little tricky, but with these tips, you can install your components in a way that minimizes the chance of breaking other products on your own.

Install the Correct Version of a Component for the Operating System and Locale

If you have operating system (OS)-specific components, make sure your setup program(s) check which OS you are using and install only the correct components. Also, you cannot give two components the same name and install them in the same directory. If you do, you overwrite the component on the second install on a dual-boot system. Note that the logo requirements recommend that you avoid installing different OS files if possible. Related to this problem is the problem caused when you install the wrong component or typelib for the locale in use, such as installing a U.S. English component on a German machine. This causes messages, labels, menus, and automation method names to be displayed in the wrong language.

Write Components to the Right Places

Avoid copying components to a system directory. An exception to this is if you are updating a system component. For that, you must use an update program provided by the group within Microsoft that maintains the component.

In general, you should copy components to the same directory that you copy the EXE. If you share components between applications, establish a shared components directory. However, it is not recommended that you share components between applications. The risks outweigh the benefits of reduced disk space consumption.

Do Not Install Older Components Over Newer Ones

Sometimes, the setup writer might not properly check the version of an installed component when deciding whether to overwrite the component or skip it. The result can be that an older version of the component is written over a newer version. Your product runs fine, but anything that depends on new features of the newer component fails. Furthermore, your product gets a reputation for breaking other products. We address the issue of whether it makes sense to overwrite components at all. But if you do overwrite them, you don’t want to overwrite a newer version.

“Copy on Reboot” If Component Is in Use

Another common mistake is to avoid dealing with the fact that you cannot overwrite a component that is in use. Instead, you have to set up the file to copy on reboot. Note that if one component is in use, you probably should set up all the components to copy on reboot. If you don’t, and if the user doesn’t reboot promptly, your new components could be mixed with the old ones.

Register Components Correctly; Take Security into Account

Sometimes setups don’t properly register COM components correctly, including the proxy and stub. Note that Windows CE requires that you also register DLLs. Note, too, that when installing DCOM components, you must be vigilant about permissions and security.

Copy Any Component That You Overwrite

It is smart to make a copy of any component that you overwrite before you overwrite it. You won’t want to put it back when you uninstall unless you’re sure that no product installed after yours will need the newer component—a difficult prediction! But by storing the component in a safe place, you make it possible for the user to fix his system if it turns out that the component you installed breaks it. You can let users know about this in the troubleshooting section of your documentation, the README file, or on your Web site. Doing this might not save a call to support, but it does at least make the problem solvable. If the component is not in use, you can move it rather than copying it. Moving is a much faster operation.

Redistribute a Self-Extracting EXE Rather Than Raw Components

If your component is redistributed by others (for instance, your component is distributed with several different products, especially third-party products), it is wise to provide a self-extracting EXE that sets up your component correctly. Make this EXE the only way that you distribute your component. (Such an EXE is also an ideal distribution package for the Web.) If you just distribute raw components, you have to rely on those who redistribute your components to get the setup just right. As we have seen, this is pretty easy to mess up.

Your EXE should support command-line switches for running silently (without a UI) and to force overwriting, even of newer components, so that product support can step users through overwriting if a problem arises.

If you need to update core components that are provided by other groups, use only the EXE that is provided by that group.

Test Setup on Real-World Systems

If you’re not careful, you can install all your setup testing on systems that already happen to have the right components installed and the right registry entries made. Be sure to test on raw systems, on all operating systems, and with popular configurations and third-party software already installed. Also, test other products to make sure they still work after you install your components.

Even Installing Correctly Does Not Always Work

Even if you follow all the preceding steps and install everything 100 percent correctly, you can still have problems caused by updating components. Why? Well, even though the new component is installed correctly, its behavior might be different enough from the old component that it breaks existing programs. Here is an example. The specification for a function says that a particular parameter must not be NULL, but the old version of the component ran fine if you passed NULL. If you enforce the spec in a new version (you might need to do this to make the component more robust), any code that passes NULL fails. Because not all programmers read the API documentation each time they write a call, this is a likely scenario.

It is also possible for a new version to introduce a bug that you simply didn’t catch in regression testing. It is even possible for clients to break as a result of purely internal improvements if they were relying on the old behavior. Typically, we assume that nothing will break when we update a component. In fact, according to Craig Wittenberg, one of the developers of COM who now works in the ComApps group at Microsoft, if you don’t have a plan for versioning your components in future releases, it is a major red flag for any component development project. In other words, before you ship Version 1, you need to have a plan for how you will update Version 1.1, Version 2, and beyond—besides how to bug-fix your updates.

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

The post Why Worry About Versioning? – Versioning Complete guide appeared first on scmGalaxy.

Best Practices in Software Configuration Management

Abstract
When deploying new SCM (software configuration management) tools,
implementers sometimes focus on perfecting fine-grained activities, while
unwittingly carrying forward poor, large-scale practices from their previous jobs or
previous tools. The result is a well-executed blunder. This paper promotes some
high-level best practices that reflect the authors’ experiences in deploying SCM.

1. Introduction
“A tool is only as good as you use it,” the saying goes. As providers of software configuration management (SCM) tools and consultants to software companies, we
are often asked for sound advice on SCM best practices – that is, how to deploy SCM software to the maximum advantage. In answering these requests we have a bounty of direct and indirect SCM experience from which to draw. The direct experience comes from having been developers and codeline managers ourselves; the indirect experience comes from customer reports of successes and failures with our product (Perforce) and other SCM tools.
The table below lists six general areas of SCM deployment, and some coarse-grained best practices within each of those areas. The following chapters explain each item.

2. The Workspace
The workspace is where engineers edit source files, build the software components they’re working on, and test and debug what they’ve built. Most SCM systems have some notion of a workspace; sometimes they are called “sandboxes”, as in Source Integrity, or “views”, as in ClearCase and Perforce. Changes to managed SCM repository files begin as changes to files in a workspace. The best practices for workspaces include:· Don’t share workspaces. A workspace should have a single purpose, such as an edit/build/test area for a single developer, or a build/test/release area for a product release. Sharing workspaces confuses people, just as sharing a desk does. Furthermore, sharing workspaces compromises the SCM system’s ability to track activity by user or task. Workspaces and the disk space they occupy are cheap; don’t waste time trying to conserve them.· Don’t work outside of managed workspaces. Your SCM system can only track work in progress when it takes place within managed workspaces. Users working outside of workspaces are beached; there’s a river of information flowing past and they’re not part of it. For instance, SCM systems generally use workspaces to facilitate some of the communication among developers working on related tasks. You can see what is happening in others’ workspaces, and they can see what’s going on in yours. If you need to take an emergency vacation, your properly managed workspace may be all you can leave behind. Use proper workspaces.
· Don’t use jello views. A file in your workspace should not change unless youexplicitly cause the change. A “jello view” is a workspace where file changes are
caused by external events beyond your control. A typical example of a jello view is a workspace built upon a tree of symbolic links to files in another workspace –
when the underlying files are updated, your workspace files change. Jello views are a source of chaos in software development. Debug symbols in executables
don’t match the source files, mysterious recompilations occur in supposedly trivial rebuilds, and debugging cycles never converge – these are just some of the
problems. Keep your workspaces firm and stable by setting them up so that users have control over when their files change.· Stay in sync with the codeline. As a developer, the quality of your work depends on how well it meshes with other peoples’ work. In other words, as changes are checked into the codeline, you should update your workspace and integrate those changes with yours. As an SCM engineer, it behooves you to make sure this workspace update operation is straightforward and unencumbered with tricky or time-consuming procedures. If developers find it fairly painless to update their workspaces, they’ll do it more frequently and integration problems won’t pile up at project deadlines.
· Check in often. Integrating your development work with other peoples’ work also requires you to check in your changes as soon as they are ready. Once you’ve
finished a development task, check in your changed files so that your work is available to others. Again, as the SCM engineer, you should set up procedures that encourage frequent check-ins. Don’t implement unduly arduous validation procedures, and don’t freeze codelines (see Branching, below). Short freezes are bearable, but long freezes compromise productivity. Much productivity can be wasted waiting for the right day (or week, or month) to submit changes.

3. The Codeline
In this context, the codeline is the canonical set of source files required to produce your software. Typically codelines are branched, and the branches evolve into variant
codelines embodying different releases. The best practices with regard to codelines are:
· Give each codeline a policy. A codeline policy specifies the fair use and permissible check-ins for the codeline, and is the essential user’s manual for
codeline SCM. For example, the policy of a development codeline should state that it isn’t for release; likewise, the policy of a release codeline should limit
changes to approved bug fixes.1 The policy can also describe how to document changes being checked in, what review is needed, what testing is required, and
the expectations of codeline stability after check-ins. A policy is a critical component for a documented, enforceable software development process, and a
codeline without a policy, from an SCM point of view, is out of control.· Give each codeline an owner. Having defined a policy for a codeline, you’ll soon
encounter special cases where the policy is inapplicable or ambiguous. Developers facing these ambiguities will turn to the person in charge of the
codeline for workarounds. When no one is in charge, developers tend to enact their own workarounds without documenting them. Or they simply procrastinate
because they don’t have enough information about the codeline to come up with a reasonable workaround. You can avoid this morass by appointing someone to
own the codeline, and to shepherd it through its useful life. With this broader objective, the codeline owner can smooth the ride over rough spots in software
development by advising developers on policy exceptions and documenting them.

· Have a mainline. A “mainline,” or “trunk,” is the branch of a codeline that evolves forever. A mainline provides an ultimate destination for almost all
changes – both maintenance fixes and new features – and represents the primary, linear evolution of a software product. Release codelines and development
codelines are branched from the mainline, and work that occurs in branches is propagated back to the mainline.

IMAGE  – 1

Figure 1 shows a mainline (called “main”), from which several release lines (“ver1”, “ver2” and “ver3”) and feature development lines (“projA”, “projb”, and
“projC”) have been branched. Developers work in the mainline or in a feature development line. The release lines are reserved for testing and critical fixes, and
are insulated from the hubbub of development. Eventually all changes submitted to the release lines and the feature development lines get merged into the
mainline. The adverse approach is to “promote” codelines; for example, to promote a development codeline to a release codeline, and branch off a new development
codeline. For example, Figure 2 shows a development codeline promoted to a release codeline (“ver1”) and branched into another development codeline
(“projA”). Each release codeline starts out as a development codeline, and development moves from codeline to codeline.

IMAGE – 2

The promotion scheme suffers from two crippling drawbacks: (1) it requires the policy of a codeline to change, which is never easy to communicate to everyone;
(2) it requires developers to relocate their work from one codeline to another, which is error-prone and time-consuming. 90% of SCM “process” is enforcing
codeline promotion to compensate for the lack of a mainline. Process is streamlined and simplified when you use a mainline model. With a
mainline, contributors’ workspaces and environments are stable for the duration of their tasks at hand, and no additional administrative overhead is incurred as
software products move forward to maturity.

4. Branching
Branching, the creation of variant codelines from other codelines, is the most problematic area of SCM. Different SCM tools support branching in markedly
different ways, and different policies require that branching be used in still more different ways. We found the following guidelines helpful when branching (and
sometimes when avoiding branching):·

Branch only when necessary. Every branch is more work – more builds, more changes to be propagated among codelines, more source file merges. If you keep this in mind every time you consider making a branch you may avoid sprouting unnecessary branches.
· Don’t copy when you mean to branch. An alternative to using your SCM tool’s branching mechanism is to copy a set of source files from one codeline and
check them in to another as new files. Don’t think that you can avoid the costs of branching by simply copying. Copying incurs all the headaches of branching –
additional entities and increased complexity – but without the benefit of your SCM system’s branching support. Don’t be fooled: even “read-only” copies
shipped off to another development group “for reference only” often return with changes made. Use your SCM system to make branches when you spin off parts
or all of a codeline.
· Branch on incompatible policy. There is one simple rule to determine if a codeline should be branched: it should be branched when its users need different
check-in policies. For example, a product release group may need a check-in policy that enforces rigorous testing, whereas a development team may need a
policy that allows frequent check-ins of partially tested changes. This policy divergence calls for a codeline branch. When one development group doesn’t
wish to see another development group’s changes, that is also a form of incompatible policy: each group should have its own branch.
· Branch late. To minimize the number of changes that need to be propagated from one branch to another, put off creating a branch as long as possible. For
example, if the mainline branch contains all the new features ready for a release, do as much testing and bug fixing in it as you can before creating a release
branch. Every bug fixed in the mainline before the release branch is created is one less change needing propagation between branches.
· Branch instead of freeze. On the other hand, if testing requires freezing a codeline, developers who have pending changes will have to sit on their changes
until the testing is complete. If this is the case, branch the codeline early enough so that developers can check in and get on with their work.

5. Change Propagation
Once you have branched codelines, you face the chore of propagating file changes across branches. This is rarely a trivial task, but there are some things you can do to
keep it manageable.
· Make original changes in the branch that has evolved the least since branching. It is much easier to merge a change from a file that is close to the common
ancestor than it is to merge a change from a file that has diverged considerably. This is because the change in the file that has diverged may be built upon
changes that are not being propagated, and those unwanted changes can confound the merge process. You can minimize the merge complexity by making
original changes in the branch that is the most stable. For example, if a release codeline is branched from a mainline, make a bug fix first in the release line and
then merge it into the mainline. If you make the bug fix in the mainline first, subsequently merging it into a release codeline may require you to back out
other, incompatible changes that aren’t meant to go into the release codeline.
· Propagate early and often. When it’s feasible to propagate a change from one branch to another (that is, if the change wouldn’t violate the target branch’s
policy), do it sooner rather than later. Postponed and batched change propagations can result in stunningly complex file merges.
· Get the right person to do the merge. The burden of change propagation can be lightened by assigning the responsibility to the engineer best prepared to resolve
file conflicts. Changes can be propagated by (a) the owner of the target files, (b) the person who make the original changes, or (c) someone else. Either (a) or (b)
will do a better job than (c).

6. Builds
A build is the business of constructing usable software from original source files. Builds are more manageable and less prone to problems when a few key practices are
observed:
· Source + tools = product. The only ingredients in a build should be source files and the tools to which they are input. Memorized procedures and yellow stickies
have no place in this equation. Given the same source files and build tools, the resulting product should always be the same. If you have rote setup procedures,
automate them in scripts. If you have manual setup steps, document them in build instructions. And document all tool specifications, including OS, compilers, include files, link libraries, make programs, and executable paths.
· Check in all original source. When software can’t be reliably reproduced from the same ingredients, chances are the ingredient list is incomplete. Frequently
overlooked ingredients are makefiles, setup scripts, build scripts, build instructions, and tool specifications. All of these are the source you build with.
Remember: source + tools = product.
· Segregate built objects from original source. Organize your builds so that the directories containing original source files are not polluted by built objects.
Original source files are those you create “from an original thought process” with a text editor, an application generator, or any other interactive tool. Built objects
are all the files that get created during your build process, including generated source files. Built objects should not go into your source code directories.
Instead, build them into a directory tree of their own. This segregation allows you to limit the scope of SCM-managed directories to those that contain only
source. It also corrals the files that tend to be large and/or expendable into one location, and simplifies disk space management for builds.
· Use common build tools. Developers, test engineers, and release engineers should all use the same build tools. Much time is wasted when a developer
cannot reproduce a problem found in testing, or when the released product varies from what was tested. Remember: source + tools = product.
· Build often. Frequent, end-to-end builds with regression testing (“sanity” builds) have two benefits: (1) they reveal integration problems introduced by check-ins,
and (2) they produce link libraries and other built objects that can be used by developers. In an ideal world, sanity builds would occur after every check-in, but
in an active codeline it’s more practical to do them at intervals, typically nightly. Every codeline branch should be subject to regular, frequent, and complete builds
and regression testing, even when product release is in the distant future.
· Keep build logs and build outputs. For any built object you produce, you should be able to look up the exact operations (e.g., complete compiler flag and link
command text) that produced the last known good version of it. Archive build outputs and logs, including source file versions (e.g., a label), tool and OS
version info, compiler outputs, intermediate files, built objects, and test results, for future reference. As large software projects evolve, components are handed
off from one group to another, and the receiving group may not be in a position to begin builds of new components immediately. When they do begin to build
new components, they will need access to previous build logs in order to diagnose the integration problems they encounter.

7. Process
It would take an entire paper, or several papers, to explore the full scope of SCM process design and implementation, and many such papers have already been written. Furthermore, your shop has specific objectives and requirements that will be reflected in the process you implement, and we do not presume to know what those are. In our experience, however, some process concepts are key to any SCM implementation:
· Track change packages. Even though each file in a codeline has its revision history, each revision in its history is only useful in the context of a set of related
files. The question “What other source files were changed along with this particular change to foo.c?” can’t be answered unless you track change
packages, or sets of files related by a logical change. Change packages, not individual file changes, are the visible manifestation of software development.
Some SCM systems track change packages for you; if yours doesn’t, write an interface that does.
· Track change package propagations. One clear benefit of tracking change packages is that it becomes very easy propagate logical changes (e.g., bug fixes)
from one codeline branch to another. However, it’s not enough to simply propagate change packages across branches; you must keep track of which
change packages have been propagated, which propagations are pending, and which codeline branches are likely donors or recipients of propagations.
Otherwise you’ll never be able to answer the question “Is the fix for bug X in the release Y codeline?” Again, some SCM systems track change package
propagations for you, whereas with others you’ll have to write your own interface to do it. Ultimately, you should never have to resort to “diffing” files to
figure out if a change package has been propagated between codelines.

· Distinguish change requests from change packages. “What to do” and “what was done” are different data entities. For example, a bug report is a “what to do”
entity and a bug fix is a “what was done” entity. Your SCM process should distinguish between the two, because in fact there can be a one-to-many
relationship between change requests and change packages.
· Give everything an owner. Every process, policy, document, product, component, codeline, branch, and task in your SCM system should have an
owner. Owners give life to these entities by representing them; an entity with an owner can grow and mature. Ownerless entities are like obstacles in an ant trail
– the ants simply march around them as if they weren’t there.
· Use living documents. The policies and procedures you implement should be described in living documents; that is, your process documentation should be as
readily available and as subject to update as your managed source code. Documents that aren’t accessible are useless; documents that aren’t updateable
are nearly so. Process documents should be accessible from all of your development environments: at your own workstation, at someone else’s
workstation, and from your machine at home. And process documents should be easily updateable, and updates should be immediately available.

8. Conclusion
Best practices in SCM, like best practices anywhere, always seem obvious once you’ve used them. The practices discussed in this paper have worked well for us, but
we recognize that no single, short document can contain them all. So we have presented the practices that offer the greatest return and yet seem to be violated more
often than not. We welcome the opportunity to improve this document, and solicit both challenges to the above practices as well as the additions of new ones.

10. References
Berczuk, Steve. “Configuration Management Patterns”, 1997. Available at
http://www.bell-labs.com/cgi-user/OrgPatterns/OrgPatterns?ConfigurationManagementPatterns.
Compton, Stephen B, Configuration Management for Software, VNR Computer
Library, Van Nostrand Reinhold, 1993.
Continuus Software Corp., “Work Area Management”, Continuus/CM: Change
Management for Software Development. Available at
http://www.continuus.com/developers/developersACE.html.
Dart, Susan, “Spectrum of Functionality in Configuration Management Systems”,
Software Engineering Institute, 1990. Available at
http://www.sei.cmu.edu/technology/case/scm/tech_rep/TR11_90/TOC_TR11_90.html

Jameson, Kevin, Multi Platform Code Management, O’Reilly & Associates, 1994
Linenbach, Terris, “Programmers’ Canvas: A pattern for source code management”
1996. Available at http://www.rahul.net/terris/ProgrammersCanvas.htm.
Lyon, David D, Practical CM, Raven Publishing, 1997
McConnell, Steve, “Best Practices: Daily Build and Smoke Test”,
IEEE Software, Vol. 13, No. 4, July 1996
van der Hoek, Andre, Hall, Richard S., Heimbigner, Dennis, and Wolf, Alexander L.,
“Software Release Management”, Proceedings of the 6th European Software
Engineering Conference, Zurich, Switzerland, 1997.

10. Author

Laura Wingerd
Perforce Software, Inc.
wingerd@perforce.com
Christopher Seiwald
Perforce Software, Inc.
seiwald@perforce.com

The post Best Practices in Software Configuration Management – SCM Best Practices Guide appeared first on scmGalaxy.

Software engineers usually find coding to be the most satisfying aspect of their job. This is easy to understand because programming is a challenging, creative activity requiring extensive technical skills. It can mean getting to “play” with state-of-the-art tools, and it provides almost instant gratification in the form of immediate feedback. Programming is the development task that most readily comes to mind when the profession of software engineering is mentioned.
That said, seasoned engineers and project managers realize that programmers are part of a larger team. All of the integral tasks, such as quality assurance and verification and validation, are behind-the-scenes activities necessary to turn standalone software into a useful and usable commodity. Software configuration management (SCM) falls into this category—it can’t achieve star status, like the latest “killer app,” but it is essential to project success. The smart software project manager highly values the individuals and tools that provide this service.
This chapter will answer the following questions about software configuration management.

What Is Software Configuration Management?
Software configuration management (SCM) is the organization of the components of a software system so that they fit together in a working order, never out of synch with each other. Those who have studied the best way to manage the configuration of software parts have more elegant responses.
Roger Pressman says that SCM is a “set of activities designed to control change by identifying the work products that are likely to change, establishing relationships among them, defining mechanisms for managing different versions of these work products, controlling the changes imposed, and auditing and reporting on the changes made.”>
We think that Pressman’s description is a better description because we often view SCM as meaning software change management.
Wayne Babich describes SCM as “the art of identifying, organizing, and controlling modifications to the software being built by a programming team. It maximizes productivity by minimizing mistakes.”>
The Software Engineering Institute says that it is necessary to establish and maintain the integrity of the products of the software project throughout the software life cycle. Activities necessary to accomplish this include identifying configuration items/units, systematically controlling changes, and maintaining the integrity and the traceability of the configuration throughout the software life cycle.
Military standards view configuration as the functional and/or physical characteristics of hardware/software as set forth in technical documentation and archives in a product. In identifying the items that need to be configured, we must remember that all project artifacts are candidates—documents, graphical models, prototypes, code, and any internal or external deliverable that can undergo change. In SW PM terminology, a configuration item might be a proposal/estimate or bid, project plan, risk management plan, quality assurance plan, CM plan itself, test plan, system requirements specification, system design document, review metric, code, test result, tool (editors, compilers, CASE), and so on. There are basic objects and aggregate objects to be configured. The number of relationships among them reflects the complexity of the configuration task.

Why Is SCM Important?
Software project managers pay attention to the planning and execution of configuration management, an integral task, because it facilitates the ability to communicate status of documents and code as well as changes that have been made to them. High-quality released software has been tested and used, making it a reusable asset and saving development costs. Reused components aren’t free, though—they require integration into new products, a difficult task without knowing exactly what they are and where they are.
CM enhances the ability to provide maintenance support necessary once the software is deployed. If software didn’t change, maintenance wouldn’t exist. Of course, changes do occur. The National Institute of Standards and Technology (NIST) says that software will be changed to adapt, perfect, or correct it. Pressman points out that new business, new customer needs, reorganizations, and budgetary or scheduling constraints may lead to software revision.
CM works for the project and the organization in other ways as well. It helps to eliminate confusion, chaos, double maintenance, the shared data problem, and the simultaneous update problem, to name but a few issues to be discussed in this chapter.

Who Is Involved in SCM?
Virtually everyone on a software project is affected by SCM. From the framers of the project plan to the final tester, we rely on it to tell us how to find the object with the latest changes. During development, when iterations are informal and frequent, little needs to be known about a change except what it is, who did it, and where it is. In deployment and baselining, changes must be prioritized, and the impact of a change upon all customers must be considered. A change control board (CCB) is the governing body for modifications after implementation.

How Can Software Configuration Be Implemented in Your Organization?
We used to say, “Make a plan and stick with it—never waffle,” and “Requirements must be frozen—how else will we know what to code?” Now, we say, “Plans are living documents—they will be in a continual state of change as project knowledge increases.” We now know that requirements are never frozen—they merge, morph, and evolve and become expanded, enhanced, and extended. As long as artifacts of software development can undergo change, we will need some method of managing the change.
Because SCM is such a key tool in improving the quality of delivered products, understanding it and how to implement it in your organization and on your projects is a critical success factor. This chapter will review SCM plan templates and provide you with a composite SCM plan template for use in any of your projects. We will cover the issues and basics for a sound software project CM system, including these:

• SCM principles
• The four basic requirements for an SCM system
• Planning and organizing for SCM
• SCM tools
• Benefits of SCM
• Path to SCM implementation

The post What Is Software Configuration Management, its importance & how to implement it? appeared first on scmGalaxy.

5 Keys to Automating Configuration Management for Application Infrastructure

One of the trends being discussed in business, among vendors and in the analyst community is the importance of automating the functions performed by IT. Growing demands by the business, tight budgets and compliance pressures together accentuate the need for IT to be more agile, efficient and responsive to business stakeholders.

Naturally, vendors rush into this environment, each touting the unique benefits of its solution set and the urgency to move forward immediately.  A key area targeted for IT automation is the area of ‘configuration management.’  As it relates to automating day to day IT functions, configuration management can mean many different things: patch management, server and network management or others.

One of the newer targets for configuration management techniques is the application layer itself.  Think about that great stack of software that comprises a contemporary J2EE application—application server, web server, database, middleware and so on—typically from different vendors.  But they all adhere to industry standards, right?  So they all must plug and play together nicely, right? Not really.

In fact to make the entire stack work effectively, there are many individual configuration files, each one with a long list of their own parameters, which need to be edited, tuned and controlled.  And because the set of software is so complex, each element is managed by its own specialist in isolation without much knowledge of the other pieces of the puzzle.

The combination of many software assets, configured manually without much knowledge of dependencies leads to predictable results.  Someone makes a change in one area while stability and performance problems show up elsewhere.  Once the problem crops up, a team of IT specialists will take hours, maybe days to track down the problem and provide the solution.  Analyst firms like Enterprise Management Associates and Forrester Research agree problems in the configuration of the application infrastructure are now one of the leading causes of downtime.

Some vendors and businesses are now focused on a comprehensive approach to tackling this problem. The five building blocks required are easy enough to understand:

1. Discovery and mapping—What application element and software assets do I have in my environment? How are they configured, item by item?

1. Change monitoring—Inform management not only about the applications that have changed, but tell them how they have changed.

1. Release Management—Sometimes called ‘provisioning,’ this relates to the ability to model configuration changes to the application infrastructure and then deploy those changes consistently across all phases of the application life-cycle.

1. Auditing and Reporting—Show the business that IT supports corporate governance initiatives, not only at the server and network level, but also at the critically important application layer.

1. Integration with the IT environment—Insure that the tools for configuration management of the application infrastructure work seamlessly with your problem management system or your configuration management database.

1.‘Discovery’ is usually where these solutions should start—as long as they do not end there!  In order to provide the IT team with a solution to managing the thousands of configuration settings in a J2EE application stack, you need to have a repository of the environment itself, a working model that reflects your application infrastructure.  With this repository of data in place, IT can then begin to comprehend how the various elements on the application layer are actually

2.‘Change Monitoring’ represents a critical component of a configuration management solution for application infrastructure.  With so many servers, instances and individual parameters, the number of configuration items quickly runs to the thousands per application.

Change monitoring must identify not only that a component changed, it must also identify exactly where the change was made and on which server(s).  Right now, IT specialists comb through text files trying to figure this out when there’s an application outage or when the application can’t make the transition out of the lab and into production. At mValent we call this “Hack and Hope”– they hack into text files, make a change and hope that it resolves the problem.  Automation tools instead can be used not only to find the change, but to find only relevant changes, ignoring unimportant differences.

Another key component of change monitoring to be addressed would be the notion of versioning and roll-back.  This is a well understood capability that exists in source code control systems and was applied by Documentum among others to the content management problem.  Now, we should focus on providing versioning and roll-back to the thousands of configuration settings which comprise a J2EE application stack.  Rather than have IT scramble to figure out what has changed when there’s an outage, let’s just re-instate a known working version of all of the configuration settings.  Then, IT can take the analysis and resolution off-line without the pressure to restore service immediately.

3. ‘Release Management’ is an important forward-looking area of a configuration management solution for application infrastructure. Right now, IT teams use a mixture of scripts and manual methods to deploy changes to application infrastructure settings. New solutions look to apply true automation to this process, insuring that configuration changes are modeled, evaluated and approved prior to deployment. Release management also validates that the process applies changes correctly. This also offers automatic deployment of changes to configuration settings and provides ‘out of the box’ templates for deploying new versions of an application server or a complete J2EE application stack.

These tasks currently take days to achieve and are error-prone.  Applying automation reduces cost and time while advancing quality.

4. ‘Audit and Reporting’ in a configuration management solution for application infrastructure refers to a capability report on the application infrastructure at a detailed level.  This would include reporting on change activity—which applications are driving the most change and causing IT execs the most headaches with SLAs.  Currently, this information exists at the server and network level, but less commonly for the applications themselves where change and upheaval is more likely.

Audit and Reporting also measures how your application infrastructure complies with recommendations, standards or your best practices for how the infrastructure should be configured.  Which assets are in compliance? Which are out of compliance?

Solutions should also measure change activity. Who made the change, Why, When, and so on.  And, it would be appropriate to measure whether changes were made according to your change process or were made outside that process.

5. ‘Integration with your IT environment’ in a configuration management solution for application infrastructure refers to the ability of the solution to fit with the other elements in the IT ecosystem, like your problem and incident management system, your Configuration Management Data Base (CMDB), a source code management system or your corporate LDAP directory.

Ideally the solution will integrate directly with these items and create a ‘closed loop’ environment for managing change to the application infrastructure.  As an example, when a ticket is opened requesting a change to the configuration setting for the web server, the solution should automate the actual change process, include the ticket number in the reason for change and report back to the incident system so that the ticket can be closed.

This eliminates yet another manual process and opportunity for error in IT management.

Benefits & Summary.  The major benefits of a configuration management solution for application infrastructure relate to reducing complexity and promoting compliance for IT.  By transforming today’s manual, error-prone tasks into a sequence of automated processes, your IT team will see benefits in:

·  Improving the productivity of your IT Infrastructure Team, by upwards of 50% and enabling them to spend more time on new initiatives rather than fire-fighting

·  Accelerating time-to-value for applications by 25%

·  Improving the quality and uptime of your applications by reducing the key elements that cause most of today’s outages

·  Providing comprehensive best practices and compliance reports to management and stakeholders.

With today’s IT struggling to meet increasing demands without headcount increases, a configuration management solution for application infrastructure offers important sources of value. configured.  More important, IT can begin to understand the dependencies between, say, how the data base is configured and its impact on the application server.  Thus, application configuration problems can be averted before they arise.

 Source: http://www.cmcrossroads.com

The post 5 Keys to Automating Configuration Management for Application Infrastructure appeared first on scmGalaxy.