Former Microsoft Visual Studio Product Manager Jim McCarthy used to say, “Successful daily builds are the heartbeat of a software project. If you do not have successful daily builds, then you have no heartbeat, and your project is dead!”
Daily builds also mark the progress being made and can indicate when a project is struggling. In addition, having a daily build can keep the programmers in “ship mode”—the attitude that the product could ship with the complete features on any given day.
A better metaphor is that the build is your product assembly line. If your assembly line is not moving or broken, your business is hurting, because you cannot turn out a product. Look at what Henry Ford did with his assembly line. Ford did not invent the car; he just perfected the way it was manufactured with the assembly line. I think the same holds true for software: By having a good build process, your company will be more efficient, successful, and profitable, and your employees will be happier.
How to Successfully Release Builds and Police Build Breaks
really like the Pottery Barn rule that was misquoted by Senator John Kerry in the second Presidential debate in September 2004. Kerry said that Colin Powell “told President Bush the Pottery Barn rule: If you break it, you fix it.” The anecdote comes from Bob Woodward’s book Plan of Attack, but Woodward actually reported that Powell privately talked with aides about the rule: “You break it, you own it.” He did not say this to the President, and it actually turns out that Pottery Barn has no such rule. Still, I think every build lab needs a poster with this rule regardless of who said it.
This leads to one of the most important rules in the build lab: The build team never fixes build breaks, regardless of how trivial the break is. That’s the developer’s responsibility. We took this a step further: The developer who breaks the build has to go to his development machine, check out the file, fix it, and then go through all the check-in process steps again.
Build Breaks Always Have the Highest Priority for Everyone
This rule means that if you are a developer and you can fix the build break, and the developer who broke the build cannot be found, you should fix it immediately. Afterward, send an e-mail to the developer and the build team explaining what you did to fix the build, and remind your co-worker that he owes you a favor.
Chris Peters, a former Microsoft vice president in the systems and applications group, used to say that people have to remember that the reason everyone works here is to ship software. That means everyone: development, testing, product support, vice presidents, administrators, and so on. If you are not moving toward a direction of shipping a product every day, you need to talk to your manager and figure out what you are supposed to be doing. Helping fix build breaks or not breaking the build in the first place is a good way to help, but don’t forget the Pottery Barn rule!
At Microsoft, developers understood that when they joined the Windows NT group, the following chain reaction would occur if they broke the build:
-
We would try to call the developer at his office.
-
If the developer did not pick up, we would call his manager and continue up the organizational ladder until we found someone who could fix the break or at least point us to someone who might be able to resolve it.
-
We would call the developer at home if it was past the 9 AM to 6 PM working hours.
To follow this track, it is important to keep a list of developers’ home telephone numbers in the build lab or make them easily accessible to everyone who is working on a project. This list is especially helpful for build breaks that occur late at night or early in the morning. With the increasing number of people working from home, this list is even more important today than it was 10 years ago.
Another way to discourage developers from breaking the build is to establish a build fine or penalty. The build fine example in the following sidenote worked well in the NT group for a couple of years.
However, don’t execute the penalty by having the engineer who broke the build run the next one. Several companies try that approach, but this is another one of those “good in theory, bad in practice” deals. What usually happens is that you end up pulling a good developer off a project until some unfortunate developer breaks the build, and then you pull that person off until… you get the picture. If you entered this unknown factor into your project manager’s Gantt chart, you would see how this can mess up the development line and build progress. It really is easier and cheaper to hire a full-time builder.
Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices