Important Configuration Definitions

  • Source code— Files written in high-level languages such as C# that need to be compiled (for example, foo.cs).
  • Source(s)— All the files involved in building a product (for example, C, CPP, VB, DOC, HTM, H, and CS). This term is used mostly as a catch-all phrase that is specific not only to source code files but to all the files that are stored in version tracking systems.
  • Codeline— A tree or branch of code that has a specific purpose, such as the mainline, release line, or hotfix line that grows collectively.
  • Mainline or trunk (“The Golden Tree”)— The main codeline of the product that contains the entire source code, document files, and anything else necessary to build and release the complete product.
  • Snapshot— A specific point in time in which the sources and build are captured and stored, usually on a release or build machine.
  • Milestone— A measurement of work items that includes a specified number of deliverables for a given project scheduled for a specified amount of time that are delivered, reviewed, and fixed to meet a high quality bar. The purpose of a milestone is to understand what is done, what is left to do, and how that fits with the given schedule and resources. To do this, the team must complete a portion of the project and review it to understand where the project is in the schedule and to reconcile what is not done with the rest of the schedule. A milestone is the best way to know how much time a portion of the project will take.
  • Code freeze— A period when the automatic updates and build processes are stopped to take the final check-ins at a milestone.
  • Public build— A build using the sources from the mainline or trunk.
  • Private build (also referred to as a sandbox build)— A build using a project component tree to build more specific pieces of the product. This is usually done prior to checking in the code to the mainline.
  • Branching— A superset of files off the mainline taken at a certain time (snapshot) that contains new developments for hotfixes or new versions. Each branch continues to grow independently or dependently on the mainline.
  • Forking— Cloning a source tree to allow controlled changes on one tree while allowing the other tree to grow at its own rate. The difference between forking and branching is that forking involves two trees, whereas branching involves just one. It is also important to note that forking or cloning makes a copy (snapshot) of the tree and does not share the history between the two trees, whereas branching does share the history.
  • Virtual Build Labs (VBLs)— A Virtual Build Lab is a build lab that is owned by a specific component or project team. The owner is responsible for propagating and integrating his code into the mainline or public build. Each VBL performs full builds and installable releases from the code in its source lines and the mainline. Although the term virtual is used in the name of the labs, don’t confuse it with Virtual PC or Virtual Machines because the labs are real physical rooms and computer boxes. It is not recommended that you use Virtual software for build machines except possibly for an occasional one-off or hotfix build. , “The Build Lab and Personnel.” There is usually a hierarchy of VBLs so that code “rolls up” to the mainline or trunk. For example, let’s say that you have a mainline, Project A is a branch off of the mainline, and Developer 1 has a branch off the project branch. Developer 1 has several branches off his branch, with each branch representing a different component of the product. If he wants to integrate one of his branches into main, he should first merge his changes with all the levels above the branch to make sure he gets all the changes. Alternatively, he can just roll the changes into main, which sits higher in the hierarchy. This will become clearer in the next couple of pages.
  • Reverse integration (RI)— The process of moving sources from one branch or tree to another that is higher in the VBL hierarchy.
  • Forward integration (FI)— The process of moving sources from one branch or tree to another that is lower in the VBL hierarchy.
  • Buddy build— A build performed on a machine other than the machine that the developer originally made changes on. This is done to validate the list of changed files so that there are no unintended consequences to the change in the mainline build.
Tagged : / / / / / / / /

Types of Builds: Developers and Project

I like to say that there are really only two types of builds: ones that work and ones that don’t. Seriously, though, when you’re shipping a product, you should consider these two different types of builds:

  • Developers’ (local machine builds)— These types of builds often happen within an editor such as Visual Studio, Emaqs, Slick, or VI. Usually, this is a fast compile/link of code that the developer is currently working on.
  • Project (central build process)— This type of build typically involves several components of an application, product, or a large project, such as Windows, or in some cases several projects included in a product, such as Microsoft Office.

The developer’s build process should be optimized for speed, but the project build process should be optimized for debugging and releases. I am talking about optimizing the process, not compiler or linker optimization switches. Although speed and debugging are important to everyone who is writing code, you must design a project build process to track build breaks and the offender(s) as quickly as possible because numerous people are waiting for a build to be released. For a developer, what seems to be most important is clicking some type of Build and Run button to make sure the code compiles without errors and then checking it in. For the build team, building without errors and having the ability to track down the person who broke the build is the most important thing.

 

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

Tagged : / / /

The Importance of a Successful Daily Build

Former Microsoft Visual Studio Product Manager Jim McCarthy used to say, “Successful daily builds are the heartbeat of a software project. If you do not have successful daily builds, then you have no heartbeat, and your project is dead!”

Daily builds also mark the progress being made and can indicate when a project is struggling. In addition, having a daily build can keep the programmers in “ship mode”—the attitude that the product could ship with the complete features on any given day.

A better metaphor is that the build is your product assembly line. If your assembly line is not moving or broken, your business is hurting, because you cannot turn out a product. Look at what Henry Ford did with his assembly line. Ford did not invent the car; he just perfected the way it was manufactured with the assembly line. I think the same holds true for software: By having a good build process, your company will be more efficient, successful, and profitable, and your employees will be happier.

How to Successfully Release Builds and Police Build Breaks

really like the Pottery Barn rule that was misquoted by Senator John Kerry in the second Presidential debate in September 2004. Kerry said that Colin Powell “told President Bush the Pottery Barn rule: If you break it, you fix it.” The anecdote comes from Bob Woodward’s book Plan of Attack, but Woodward actually reported that Powell privately talked with aides about the rule: “You break it, you own it.” He did not say this to the President, and it actually turns out that Pottery Barn has no such rule. Still, I think every build lab needs a poster with this rule regardless of who said it.

This leads to one of the most important rules in the build lab: The build team never fixes build breaks, regardless of how trivial the break is. That’s the developer’s responsibility. We took this a step further: The developer who breaks the build has to go to his development machine, check out the file, fix it, and then go through all the check-in process steps again.

Build Breaks Always Have the Highest Priority for Everyone

This rule means that if you are a developer and you can fix the build break, and the developer who broke the build cannot be found, you should fix it immediately. Afterward, send an e-mail to the developer and the build team explaining what you did to fix the build, and remind your co-worker that he owes you a favor.

Chris Peters, a former Microsoft vice president in the systems and applications group, used to say that people have to remember that the reason everyone works here is to ship software. That means everyone: development, testing, product support, vice presidents, administrators, and so on. If you are not moving toward a direction of shipping a product every day, you need to talk to your manager and figure out what you are supposed to be doing. Helping fix build breaks or not breaking the build in the first place is a good way to help, but don’t forget the Pottery Barn rule!

At Microsoft, developers understood that when they joined the Windows NT group, the following chain reaction would occur if they broke the build:

  1. We would try to call the developer at his office.

  2. If the developer did not pick up, we would call his manager and continue up the organizational ladder until we found someone who could fix the break or at least point us to someone who might be able to resolve it.

  3. We would call the developer at home if it was past the 9 AM to 6 PM working hours.

To follow this track, it is important to keep a list of developers’ home telephone numbers in the build lab or make them easily accessible to everyone who is working on a project. This list is especially helpful for build breaks that occur late at night or early in the morning. With the increasing number of people working from home, this list is even more important today than it was 10 years ago.

Another way to discourage developers from breaking the build is to establish a build fine or penalty. The build fine example in the following sidenote worked well in the NT group for a couple of years.

However, don’t execute the penalty by having the engineer who broke the build run the next one. Several companies try that approach, but this is another one of those “good in theory, bad in practice” deals. What usually happens is that you end up pulling a good developer off a project until some unfortunate developer breaks the build, and then you pull that person off until… you get the picture. If you entered this unknown factor into your project manager’s Gantt chart, you would see how this can mess up the development line and build progress. It really is easier and cheaper to hire a full-time builder.

 

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

Tagged : / / /

Build Archive Policy

sgoldstein created the topic: Build Archive Policy
I wanted to get some input from the SCM community regarding build archiving. When using either a CI or nightly build approach, a number of builds are generated during a release. How do people generally define a retention policy for retaining builds on disk? Once the retention policy dictates that a build should be removed from disk, what happens to it? Do people back it up to tape or some other media? Do people simply delete it entirely?

I ask because there are situations in which, for example, a bug is filed against a particular build and the dev team may want to go back an investigate. Typically, we’ve been archiving to tape, but it’s very time consuming and I don’t feel that the value is outweighing the cost.

Thanks.

rajeshkumar replied the topic: Re:Build Archive Policy
It’s all about process you follow and what release cycle you have..Dev team doesn’t need to refer old build until some milestone has been achieved as part of build. So as a Build engineer you need to know all the milestone achieved and which build CI# and revision number.
you can write a script which will store a information about revision number associated with the build and Build number, this number can be referenced in the future to get same build using CI tools
Scenario two: here I do follow some process. Like

I always clean up incremental build.

I do escrow audit after any patch, major or minor release.

Base line after any milestone met and documentation

Keep more than one year build only when if its matching with some milestone
Regards,
Rajesh Kumar
Twitt me @ twitter.com/RajeshKumarIn

Tagged :

Build step ‘Execute shell’ marked build as failure

scmuser created the topic: Build step ‘Execute shell’ marked build as failure
Build step ‘Execute shell’ marked build as failure

How to resolve this issues? I am building this in slave?

[QA-QBO-Square-e2e-regression] $ /bin/sh -xe /tmp/hudson2656830909630025820.sh
+ cat
/tmp/hudson2656830909630025820.sh: 2: /tmp/hudson2656830909630025820.sh: cat: not found
Build step ‘Execute shell’ marked build as failure

mastinder@gmail.com replied the topic: Build step ‘Execute shell’ marked build as failure
Hi,

use -li option at the end of shebang and it should work

#!/bin/zsh -li
set -xv
ear_path=/cygdrive/e/mas2/maven/repository/ie/aib/ems/masii-ear

Tagged :

Modified files after each build

setu.sin created the topic: Modified files after each build
Hi All,

I need to send the modified files to an external API after each build in jenkins. I can see the recently changed files in jenkins that pulls from git commit but can you please tell me where it is stored so that i can use that to push the info to another release management tool we have.

Thanks

rajeshkumar replied the topic: Modified files after each build
Question is not very much clear or you are lacking in understanding of How git works? When you say modified file, means the modification is being done in jenkins workspace as part of build process or Latest changes which was checked out by Jenkins from git??? Need to elaborate more the scnario
Regards,
Rajesh Kumar
Twitt me @ twitter.com/RajeshKumarIn

setu.sin replied the topic: Modified files after each build
Hi Rajesh,

With each build, there are some code changes checked into git. After i run a build, i get the list of those code changes by clicking on recent changes which shows the commits that were performed prior to this build. I need to display this change on another tool so I am trying to understand where these changeset it stored so that i can pass it as a string to the release management tool API.

Thanks

rajeshkumar replied the topic: Modified files after each build
Jenkins store the last checked out code. Post build, jenkins “Recent Changes” section with the help of git plugins display the output of

git log [first]..[second] –pretty=oneline

[first] and [second] can be SHA’s, branches or tags.
Regards,
Rajesh Kumar
Twitt me @ twitter.com/RajeshKumarIn

rajeshkumar replied the topic: Modified files after each build
Jenkins store the last checked out code. Post build, jenkins “Recent Changes” section with the help of git plugins display the output of

git log [first]..[second] –pretty=oneline

[first] and [second] can be SHA’s, branches or tags.
Regards,
Rajesh Kumar
Twitt me @ twitter.com/RajeshKumarIn

Tagged :

Build WixUtilExtension.dll

applicationPackaging created the topic: Build WixUtilExtension.dll
I want to build WixUtilExtension.dll from wix sources.
So first i tried to build util.wixlib , but that has references to caerr.wxi, wixca.dll and other files.
Not , I am trying to build wixca.dll from wix sources. But haven’t been able to do so yet. I did go through this blog, but this was not of much help to me.

neilsleightholm.blogspot.com/search?q=build+wix

If someone could give me some pointers.
I want to be able to build it through Visual Studio and not using Nant.
Also if someone could tell me the dependencies of wixca.(one of them is wixcautil ).
Any help would be useful.

Tagged :

Moving build system from one machine to another.

msiexpert created the topic: Moving build system from one machine to another.
Till now we were building our software on a development build machine.We need to move it to a production build machine.We have bought a studio version of InstallAware 9.Will we have any problems with licensing if uninstall the InstallAware from dev setup and install it on production setup? Please help.

Tagged :

How to build when a change is pushed to Bitbucket

bitbucket-tutorials

 

Bitbucket plugin is designed to offer integration between Bitbucket and Jenkins.

 

It exposes a single URI endpoint that you can add as a WebHook within each Bitbucket project you wish to integrate with. This single endpoint receives a full data payload from Bitbucket upon push (see their documentation), triggering compatible jobs to build based on changed repository/branch.

 

Step 1 – Install “Bitbucket Plugin” at your Jenkins

 

Step 2 – Add a normal Post as Hook to your Bitbucket repository (Settings -> Hooks) and use following url:

 

https://YOUR.JENKINS.SERVER:PORT/bitbucket-hook/
and if you have setup authentication on jenkins then URL must be like

 

https://USERNAME:PASSWORD@YOUR.JENKINS.SERVER:PORT/bitbucket-hook/

 

Step 3 – Configure your Jenkins project as follows:

 

Step 4 – Under build trigger enable Build when a change is pushed to BitBucket

 

Step 5 – under Source Code Management select GIT; enter your credentials and define Branches to build (like **feature/*)

 

Note 1 – Make sure to include the slash (‘/’) on the end of the URL or the hook won’t work.

 

Note 2 – Please read the BitBucket Plugin info page as well https://wiki.jenkins.io/display/JENKINS/BitBucket+Plugin

 

Reference 1  
Reference 2 – Login issues with Jenkins url
Tagged : / / / / / /

Top 5 Build Management Tools

top-5-build-management-tools
These days in software industry the process of software development very much rely upon best practices of various tools. The software development teams use various tools like project management, release management , test management and various others. As we already discussed about these tools in our previous posts. But, today we are going to discuss about Build management tool. This is one of the most important tool which required in any kind of software development. In this article we are going to discuss about Top Build Management Tools.
But before that let’s have a quick overview on Build management.
Build management is actually a process of collecting all the components in a software release, performing all the automated tasks to compile, build and test the system and then deploy onto the development and testing environments in preparation for staging. It has become an important part of software development in testing process.
But you can not simply do build management without tools. it is also important to ensure that tools are selected properly and thoroughly so that each of them provides the desired service.
So, without wasting any time let’s have a look on top 5 build management tools
1. Apache Maven
 Apache Maven
Maven is an open source build management tool which is distributed under Apache License. It is basically used for Java projects. let’s have a look on key feature of Maven tool
Key Features
  • Open source
  • Based on Project Object Model or POM
  • Release management and distribution publication
  • Coherent site of project information
  • Instant access to new features
  • Extensible with plugins
  • Works easily with multiple projects simultaneously
  • Simple project setup

2. Gradle

 

 Gradle

Gradle is also an open source build management tool which is distributed under Apache License. It is written in java and groovy supports cross platform. Let’s have a look on key features

Key Features
  • Open source
  • Designed for multiple projects
  • Supports incremental builds
  • Rich API
  • Mature ecosystem of plugins
  • Ease of migration
  • First build integration tool
  • Declarative builds and build-by-convention

3. Apache Ant

 

Apache Ant
Apache Ant is also an open source build automation tool which is distributed under Apache License. It is also a Java based build tool. Let’s have a look on key features
Key Features
  • open source
  • Ease of Use
  • Independent Platform
  • Can execute test scripts and test suites
  • Can copy files to at different locations
  • Supports Junit 3, Junit 4, Testing etc.
  • Able to compile java based applications
  • Can check out the code from version control system (SVN, GIT, CVS etc).
4. MSBuild
 MSBuild
MsBuild or Microsoft build tools as its name indicates it belongs to Microsoft which is written in C# and supports .Net framework and available under MIT license.
Key features
  • Able to build Visual Studio projects without Visual Studio IDE installed
  • Now bundled with Visual Studio
  • Multitargeting
  • Description language – XML
5. FinalBuilder
 FinalBuilder
Finalbuilder is a build tool which supports Windows platform and it is developed by VSoft Technologies in the year 2000. It is available under Proprietary license.
Key Features
  • It can present your build process in a structured manner
  • With FinalBuilder you don’t need to edit xml, or  write scripts
  • Allows builds to be scheduled to run daily, weekly or whenever you wish
  • Extensive library of pre-written actions to automate every common task in build process
  • Integrated Debugging
  • Detailed Logging
  • Version Control Integration
Do you agree with this list? If not than feel free to respond in the comment box with your own take on the top build management tools. One more thing, I would like to add here, if you need help to learn all these build tools or DevOps courses than scmGalaxy can help you in this. scmGalaxy is a community of DevOps professionals who are well experienced in this domain. So, feel free to reach us.
Tagged : / / / / / / / / / / / / /