Types of Build

follow are Types of Build…

Distributed processing Build:
Parallel processing build
Multi-platform builds
Multi-language builds
Dedicated builds

build types

Tagged : / / / / / / /

Types of Build in Remote Agent

Types of Build in Remote Agent

Build Management tools has one one capability where they can share the infrasture to build the product in less time with the help of Remote Agent Machine.

There are following types of build in remote Agent.

Distributed processing Build:
distributed build is one in which individual steps in the Workflow are sent to be executed on multiple machines.  In doing this you are able to leverage more machine power instead of attempting to run the entire Workflow on a single machine.

Parallel processing build
Remote Agents also support parallel processing.  A Parallel process executes Workflows for each unique stage of the development lifecycle including continuous integration builds for developers, pre-production builds for testers and emergency builds for production control. A Remote Agent can be configured to execute builds according to the location that the binaries will be distributed.

Multi-platform builds
Workflows can be configured to call Remote Agents that are  running different operating systems This allows you to execute a Workflow that builds the application across multiple operating systems or build specific components of the application on the appropriate machine.   For example, a Workflow that needs to build Windows .Net components as well as AIX Oracle back-end components would use two Remote Agents one for Windows and one for AIX.

Multi-language builds 
Workflows can be configured to call Remote Agents that are  running different languages.  This allows you to execute a Workflow that builds the application across multiple operating systems or build specific components of the application on the appropriate machine.   For example, a Workflow that needs to build Windows .Net components as well as AIX Oracle back-end components would use two Remote Agents one for Windows and one for AIX.
Dedicated builds

Remote Agents can also be used as ‘dedicated build machines’.  A dedicated “Build Machine” is often bigger and faster than a regular desktop machine.  The dedicated build machine would be configured as a Remote Agent in which Workflows could be executed in order to improve the workflow processing.  In addition, if you are using Meister’s Build Automation, a dedicated machine with multi-processing power can be used to manage the calls to the compilers and linkers and accelerate the building of C and Java applications.

 

image

Tagged : / / / /

Steps for a complete clean build

Following are the steps for a complete clean build:

1.
Build project (compilation)—In the build phase, the build system compiles operating system (OS) component source files and produces libraries. The basic unit of componentization in Windows CE is the library—components are not conditionally compiled. Because of this, components can be mixed and matched without worrying about changes in their behavior.

2.
Link project—During the link phase, the build system attempts to build all target modules. Modules are drivers and executables produced from Windows CE components. In CE 4.0 and later, you can select modules via sysgen environment variables. For example, the “common” project’s modules are listed in CE_MODULES, the DirectX project’s modules are listed in DIRECTX_MODULES, Internet Explorer’s modules are listed in IE_MODULES, and so on.

Microsoft introduced the separation of the build phase and the link phase in Windows CE .NET. Because the operating system was getting more and more complex, linking drivers and other components during the build phase could possibly cause hard-to-diagnose crashes at runtime because coredll entry points that were present during the build phase (which occurs prior to componentization) might not be present in an OEM’s final platform.

3.
Copy/filter project (headers and libraries)—The Copy/Filter phase of system generation is responsible for moving parts of the operating system to the target project’s cesysgen directory. Note: only the components of the OS that the OEM has selected are moved. In addition, header files and various configuration files such as common.bib and common.reg are “filtered” to remove the parts that are unrelated to the OEM’s selected components. The copy/filter is performed at the same time as linking.

4.
Post-process project (miscellaneous post-sysgen cleanup)—The “postproc” sysgen target provides the build system with a mechanism to do some work after most of the system has been generated. Although the post-process phase is important for the small number of OEMs who use it, most developers don’t do much with it.

5.
Platform sysgen—If an OEM wants to write his platform in such a way that it can be used with any selection of OS components, he immediately runs into a problem. Some of the drivers, Control Panel applets, or applications in the platform directory might depend on unselected components. When these source files are built, there are compilation or linker errors because header files or coredll entry points are missing.

The platform sysgen step helps address this problem by checking for a platform sysgen makefile.

6.
Build platform—This phase consists of running a clean build on all projects and building only the projects that sysgen settings specify.

7.
Create release directory—After all the OS components and the platform have been compiled and linked, you need to assemble all the appropriate binaries and configuration files into one place so that you can combine them into a downloadable image. You can use a batch file to perform this step.

8.
Create downloadable image—After you populate the release directory with the appropriate binaries, the next step is to create a binary file that is suitable for flashing or downloading to your device’s RAM. Use the makeimg command for this step.
Tagged : / / / /

Types of Builds: Developers and Project

I like to say that there are really only two types of builds: ones that work and ones that don’t. Seriously, though, when you’re shipping a product, you should consider these two different types of builds:

  • Developers’ (local machine builds)— These types of builds often happen within an editor such as Visual Studio, Emaqs, Slick, or VI. Usually, this is a fast compile/link of code that the developer is currently working on.
  • Project (central build process)— This type of build typically involves several components of an application, product, or a large project, such as Windows, or in some cases several projects included in a product, such as Microsoft Office.

The developer’s build process should be optimized for speed, but the project build process should be optimized for debugging and releases. I am talking about optimizing the process, not compiler or linker optimization switches. Although speed and debugging are important to everyone who is writing code, you must design a project build process to track build breaks and the offender(s) as quickly as possible because numerous people are waiting for a build to be released. For a developer, what seems to be most important is clicking some type of Build and Run button to make sure the code compiles without errors and then checking it in. For the build team, building without errors and having the ability to track down the person who broke the build is the most important thing.

 

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

Tagged : / / /

The Importance of a Successful Daily Build

Former Microsoft Visual Studio Product Manager Jim McCarthy used to say, “Successful daily builds are the heartbeat of a software project. If you do not have successful daily builds, then you have no heartbeat, and your project is dead!”

Daily builds also mark the progress being made and can indicate when a project is struggling. In addition, having a daily build can keep the programmers in “ship mode”—the attitude that the product could ship with the complete features on any given day.

A better metaphor is that the build is your product assembly line. If your assembly line is not moving or broken, your business is hurting, because you cannot turn out a product. Look at what Henry Ford did with his assembly line. Ford did not invent the car; he just perfected the way it was manufactured with the assembly line. I think the same holds true for software: By having a good build process, your company will be more efficient, successful, and profitable, and your employees will be happier.

How to Successfully Release Builds and Police Build Breaks

really like the Pottery Barn rule that was misquoted by Senator John Kerry in the second Presidential debate in September 2004. Kerry said that Colin Powell “told President Bush the Pottery Barn rule: If you break it, you fix it.” The anecdote comes from Bob Woodward’s book Plan of Attack, but Woodward actually reported that Powell privately talked with aides about the rule: “You break it, you own it.” He did not say this to the President, and it actually turns out that Pottery Barn has no such rule. Still, I think every build lab needs a poster with this rule regardless of who said it.

This leads to one of the most important rules in the build lab: The build team never fixes build breaks, regardless of how trivial the break is. That’s the developer’s responsibility. We took this a step further: The developer who breaks the build has to go to his development machine, check out the file, fix it, and then go through all the check-in process steps again.

Build Breaks Always Have the Highest Priority for Everyone

This rule means that if you are a developer and you can fix the build break, and the developer who broke the build cannot be found, you should fix it immediately. Afterward, send an e-mail to the developer and the build team explaining what you did to fix the build, and remind your co-worker that he owes you a favor.

Chris Peters, a former Microsoft vice president in the systems and applications group, used to say that people have to remember that the reason everyone works here is to ship software. That means everyone: development, testing, product support, vice presidents, administrators, and so on. If you are not moving toward a direction of shipping a product every day, you need to talk to your manager and figure out what you are supposed to be doing. Helping fix build breaks or not breaking the build in the first place is a good way to help, but don’t forget the Pottery Barn rule!

At Microsoft, developers understood that when they joined the Windows NT group, the following chain reaction would occur if they broke the build:

  1. We would try to call the developer at his office.

  2. If the developer did not pick up, we would call his manager and continue up the organizational ladder until we found someone who could fix the break or at least point us to someone who might be able to resolve it.

  3. We would call the developer at home if it was past the 9 AM to 6 PM working hours.

To follow this track, it is important to keep a list of developers’ home telephone numbers in the build lab or make them easily accessible to everyone who is working on a project. This list is especially helpful for build breaks that occur late at night or early in the morning. With the increasing number of people working from home, this list is even more important today than it was 10 years ago.

Another way to discourage developers from breaking the build is to establish a build fine or penalty. The build fine example in the following sidenote worked well in the NT group for a couple of years.

However, don’t execute the penalty by having the engineer who broke the build run the next one. Several companies try that approach, but this is another one of those “good in theory, bad in practice” deals. What usually happens is that you end up pulling a good developer off a project until some unfortunate developer breaks the build, and then you pull that person off until… you get the picture. If you entered this unknown factor into your project manager’s Gantt chart, you would see how this can mess up the development line and build progress. It really is easier and cheaper to hire a full-time builder.

 

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

Tagged : / / /

Performance Optimization of Build Server | Performance Optimization Guide

build-server

Performance Optimization Checklist of Build Servers

Build Infrastructure Level

1.  Do you really need to build all source code or only the part of code which has changes?

2. Project should be divided into multiple modules/component which can be build independently and should be integrated when it needed.

3. Understand technology and make use of performance related features which will help you. e.g make, visual studio and maven has some feature which help us to compile only code which was changed also, some performance tuning features should be explored.

4. Compile dependent project only there are changes. no need to compile all the time.

5. Distribute application in non-compiled and to be compiled files. e.g src, docs and copy only src to comile. docs files cane be used while creating installer.

6. Plan for disk usage growth upfront.

Application Server level

  1. Apache Server optimization and proper log setting
  2. Disabling unused modules and component which will help performance.
  3. Follow best practices of Module Configuration, Apache Server Configuration etc

 System level

  1. Check if any background task/services is running which is consuming cpu/memory and can be  stopped and not relevant.
  2. Keep system uptodate
  3. Avoid using build server for any other task – If you must edit components or install software, build or edit the components on another machine and copy them over.
  4. Use high speen drive for i/o operations

Build Server application level

http://www.cloudbees.com/sites/default/files/whitepapers/7WaysToOptimizeJenkins.pdf

  1. Keep log files maintenance and truncate regularly. have limited build records.
  2. Move build to document server. not to keep in build server
  3. Take advantage of distribute build server using agents/slaves/nodes etc
Tagged : / / / / / / / / / / / / / / /

More Important Build Definitions | Different Build Definitions | Build Terminologies

build-definitions

More Important Build Definitions

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

It is also important for groups or teams to define these terms on a project-wide basis so that everyone is clear on what he is getting when a build is released.

Pre-build— Steps taken or tools run on code before the build is run to ensure zero build errors. Also involved are necessary steps to prepare the build and release machines for the daily build, such as checking for appropriate disk space.

Post-build— Includes scripts that are run to ensure that the proper build verification tests (BVTs) are run. This also includes security tests to make sure the correct code was built and nothing was fused into the build.

Clean build— Deleting all obj files, resource files, precompiled headers, generated import libraries, or other byproducts of the build process. I like to call this cleaning up the “build turds.” This is the first part of a clean build definition. Most of the time, build tools such as NMake.exe or DevEnv.exe handle this procedure automatically, but sometimes you have to specify the file extensions that need to be cleaned up. The second part of a clean build definition is rebuilding every component and every piece of code in a project. Basically the perfect clean build would be building on a build machine with the operating system and all build tools freshly installed.

Incremental build— The secret to getting out a daily build to the test team, regardless of circumstances, is to perform incremental builds instead of daily clean builds. This is also the best way that you can maintain quality and a known state of a build. An incremental build includes only the code of the source tree that has changed since the previous build. As you can guess, the build time needed for an incremental build is just a fraction of what a clean build takes.

Continuous integration build— This term is borrowed from the extreme programming (XP) practice. It means that software is built and tested several times per day as opposed to the more traditional daily builds. A typical setup is to perform a build every time a code check-in occurs.

Build break— In the simplest definition, a build break is when a compiler, linker, or other software development tool (such as a help file generator) outputs an error caused by the source code it was run against.

Build defect— This type of problem does not generate an error during the build process; however, something is checked into the source tree that breaks another component when the application is run. A build break is sometimes referred to or subclassed as a build defect.

Last known good (LKG) or internal developers workstation (IDW) builds— These terms are used as markers to indicate that the build has reached a certain quality assurance criterion and that it contains new high-priority fixes that are critical to the next baseline of the shipping code. The term LKG originated in the Visual Studio team, and IDW came from the Windows NT organization. LKG seems to be the more popular term at Microsoft.

Tagged : / / / / / / / / / / / / / / / / / /

How we reduced build time from 8 hours to 1 hour ? – Complete Guide

hudson-master-slave-setup

Situation

  1. For one of our clients, Build is taking 8 hours and nightly build is failing frequently.
  2. Test case execution is consuming more time than the compilation.
  3. Low confidence levels for developers on nightly builds and subsequently during integration cycle.

Approach

  1. We reviewed the whole set of build scripts being used and test case suite to check how we can refatcor them.
  2. Verified the build infrastructure i.e., build server capacity from hardware point of view.
  3. Feasibility study done to check how a CI environment help in giving faster feedback on the quality.


Solution

  1. Refactoring done for build scripts to make modules compilation parallel (wherever possible).
  2. Some test cases are selected (which don’t require external services like database) to run as part of the build.
  3. CI environment (using open source cruise control) is established to run builds per commit and execute only above set of test cases as part of the build.
  4. Set up low-cost remote agent machine which will run remaining test suite once per day.

Results

  1. Build took only 1 hour and gave faster feedback to dev team.
  2. CI environment provided good confidence to dev team.
  3. Team is able to implement more best practices.
  4. Effective use of existing build  infrastructure (hardware) ensured no further cost is required.
  5. This helped the client in ensuring smooth release management.

 

Tagged : / / / / / / / / / / / / / /

Power Point PPT: Build And Automation

build-and-automation

Power Point PPT: Build And Automation

Tagged : / / / / / / / / / / / /

Build Management Recommendation – Build Management Guidance

build-management-recommendation

Recommendation from Microsoft’s Software Configuration Management Best Practices

Build Management Recommendation

Recommendation for Defining a Build

  • Define terms in your development process, and keep a glossary of them on an internal build Web page. If you like, standardize on the definitions in this chapter.
  • Clean build your complete product at least once per week, or every day if possible.
  • Use incremental builds on a daily basis if clean builds are not possible or practical.
  • Start charting the quality of your product, and post it where everyone involved in the project can see it.
  • Release LKG (or IDW) builds weekly; then switch to daily releases toward the end of the shipping cycle.
  • Follow the Software Development Flow diagram.

Recommendation for Source Tree Configuration for Multiple Sites and Parallel (Multi-Version) Development Work

  • Create the mainline (public) and virtual build labs (private) codelines.
  • Make sure the mainline is pristine and always buildable and consumable. Create shippable bits on a daily basis. Use consistent, reliable builds.
  • Build private branches in parallel with the main build at a frequency set by the CBT.
  • Use consistent reverse and forward integration criteria across teams.
  • Be aware that dev check-ins are normally made only into a private branch or tree, not the mainline.
  • Know that check-ins into a private branch are only reverse integrated (RId) into main when stringent, division-wide criteria are met.
  • Use atomic check-ins (RI) from private into main. Atomic means all or nothing. You can back out changes if needed.
  • Make project teams accountable for their check-ins, and empower them to control their build process with help from the CBT.
  • Configure the public/private source so that multisite or parallel development works.
  • Optimize the source tree or branch structure so that you have only one branch per component of your product.

Recommendation for Daily, Not Nightly, Builds

  • Hire a consulting firm to come in and review your processes and tools.
  • Start your build during the day, not during the evening.
  • Publish the build schedule on an internal Web site.
  • Release daily builds as sure as the sun comes up, but make sure they are quality, usable builds. Don’t just go through the motions.
  • Discourage build breaks by creating and enforcing consequences.

Recommendation for The Build Lab and Personnel

  • Set up a build lab if you do not already have one.
  • Purchase all necessary hardware, and do not skimp on quality or number of machines.
  • Keep the lab secure.
  • Show a lot of appreciation for your build team. They have a difficult job that requires a special skill set and personality to be successful.

Recommendation for Build Tools and Technologies

  • Use command-line builds for the central build process.
  • Use Make or ANT for non-Microsoft platforms or tools.
  • Use MSBuild for .NET builds.
  • Use VCBuild for non-.NET builds.
  • Write your scripts in an easy language such as Perl or Batch files.
  • Learn XML because it is ubiquitous.

Recommendation for SNAP Builds—aka Integration Builds

  • Develop your own integration build tool to be used as a precheck-in build test to the golden source trees, or wait for Microsoft to release a tool similar to this one. Keep the points in this chapter in mind when you’re building this “silver bullet.”
  • If you deploy a Virtual Build Lab process, require that the VBLs use a SNAP system to stay in sync with the Central Build Lab. This is the whole concept of an integration build system such as this.
  • Do not rely on the SNAP system as your mainline build tool. Although I mention that some groups at Microsoft use this tool as their Central Build Team build tool, this can be a little problematic because the check-in queues can get really backed up.
  • Understand that no magic tool is out there to do your work for you when it comes to builds and merging code. But SNAP is a good tool, and some groups at Microsoft cannot live without it.
  • Make sure you have all the other processes down in this book before trying to roll out a SNAP system. These processes include source tree configuration, build schedules, versioning, build lab, and so on.

Recommendation for The Build Environment

Here is a short list of what the build environment example in this chapter is about. If everyone is using the same batch files to launch a build environment, re-creating a build will be less painful.

  • Keep the build environment consistent, and control it through the Central Build Team’s intranet Web page.
  • Use batch file commands similar to the ones used in the examples in this chapter.
  • Enforce through checks and balances in the batch files that everyone is using the published project build environment.

Recommendation for Versioning

What I recommend is to read and re-read this chapter to make sure you fully understand everything in it and to grasp the importance of reliable versioning. Here is a short list of what you need to do:

  • Use a four-part number separated by periods for your file version string.
  • Increment your build number before starting each build.
  • Avoid the use of dates in the build number.
  • Use your build number for the label in sources you just built.
  • Don’t try to include product marketing versions in your file version string.
  • For .NET programmers, don’t link assembly versioning to file versioning.
  • During your setup program, do the following:
    • Check for the OS that you are installing on.
    • Avoid copying anything to the system directory.
    • Copy all components to the same directory as the executable.
    • Avoid installing older components over newer ones.
    • Make a copy of any component you overwrite.
    • Use a self-extracting executable to update.

Recommendation for Build Security

With all the talk about security on the Internet and in the applications that are out there, we must not forget about keeping company “jewels” safe. Here are some recommendations that were covered in this chapter:

  • At a minimum, use the four-layer approach talked about in detail in this chapter:
    • Physical security— Doors, locks, cameras, and so on.
    • Tracking source changes— The build process.
    • Binary/release bits assurance— The tools process.
    • IT infrastructure— Company-wide policy and security.
  • Consider the .NET platform as a means of security.
  • Look into software restriction policies that are in Microsoft Windows XP and Windows Server 2003.
  • Start worrying about security before a breach occurs.

Recommendation for Building Managed Code

You will find that building projects for the .NET Framework is a bit different than the classic “unmanaged code builds” that have been around before Web services were ever dreamed up. I went over the parts of building .NET code that tend to trip people in this chapter; the following is a quick list of recommendations:

  • If you build managed code, learn the basic terms of the .NET Framework and the compilation process explained in this chapter.
  • Use delayed signing when developing your project to avoid having to sign the assemblies in conjunction with your daily build.
  • Understand the risk of exposing your developer’s machine to external attacks because of the skip verification list that is created when delaying signing.
  • Decide what is the most practical way of setting up your solution files for your .NET projects. Then enforce whatever policy you come up with through your CBT.

Recommendation for International Builds

Here is what is done at most of the bigger groups at Microsoft and is thus our preferred way of internationalizing our products.

  • Write single binary code as the first step toward truly world-ready products.
  • Implementing a multilingual user interface that allows users to switch between all supported languages is the next logical step.
  • Write Unicode-aware code and create satellite DLLs for your language resources to make these goals much easier to achieve.

Recommendation for Build Verification Tests and Smoke Tests

Some of the general points of the chapter are:

  • Run Visual File Information and VerCheck on each build.
  • Establish a quality bar that the build will be measured against
  • Let the test team own the BVTs, but let the build team run them.
  • Automate BVTs and smoke tests.
  • Track BVT results on a public intranet page.
  • Know the difference between BVTs and smoke tests.
  • Always have the testers or developers run the smoke tests, not the build team.

Recommendation for Building Setup

To improve your setup reliability the following should be done:

  • Decide which setup tool you will use: Wise, InstallShield, WiX, or another brand.
  • Track all files in your product in a spreadsheet or, better yet, a database. If you are using WiX, the files are listed in the .wxs file.
  • Build setup every day, and practice deploying your product to test machines every day. Do not release the build until setup has been created successfully.
  • Start pushing the setup responsibility back to the developers who own the modules.

Recommendation for Ship It!

The following list should be done as the final steps to shipping a great product:

  • Define what and when shipping is for your group. Follow Jim McCarthy’s Rule 21.
  • Start restructuring source trees before you ship, not after.
  • Establish a standard release process for your group.
  • Release a new build process after you release.
  • Add the shipping process time to the product schedule.

Recommendation for Customer Service and Support

From reading this chapter you should understand why you need to:

  • Create and invest in a strong support organization.
  • Make sure your support group has good paths back to the developers but are not intrusive to them doing their work writing code.
  • Outline specific escalation paths in support that give useful feedback to the product group.
  • Involve your support group as early as possible in the WAR meetings to discuss impacts.

Recommendation for Managing Hotfixes and Service Packs

A lot in this chapter was specific to VSS, but I think when you read this chapter you can draw parallels to the source code control tool you use and just copy the basic architect. These recommendations can be followed with any source code control tool but is more specific to VSS.

  • Use sharing to reuse common code modules.
  • Use labeling to isolate and mark different builds.
  • Use share and pin with branching to create new minor releases.
  • Use cloning to create new major releases.
  • Remember: Sharing is the first step of branching.

Recommendation for Suggestions to Change Your Corporate or Group Culture

  • Follow the seven suggestions to change or create a successful culture:
    • Involve as high a level of management as you can when rolling out new processes or tools. In fact, have the executive send the e-mail on the new announcement.
    • Hire a consulting firm to come in and perform an analysis.
    • Match company values to what you are trying to accomplish.
    • Do the math.
    • Not worrying about getting fired gives you incredible power.
    • Never take no from someone who does not have the power to say yes.
    • Publish policies, processes, and tools on the intranet!
  • If you can’t get 100 percent behind what you are doing, you should find something else to do or somewhere else to do it.

Recommendation for Future Build Tools from Microsoft

Looking into a crystal ball that predicts the future, you should see:

  • Adopt MSBuild as soon as possible.
  • Start researching VSTS and see if there are tools you can implement in your current build process and future processes.

 

 

 

Tagged : / / / / / / / / / / / / / / / / /