Recovery Procedures in Perforce Server

Recovery Procedures in Perforce Server

If the database files become corrupted or lost, either because of disk errors, a hardware failure such as a disk crash, the database can be recreated with your stored checkpoint and journal. it can at least provide a general guideline for recovery from the two most common situations, specifically:

1. corruption of your Perforce database only, without damage to your versioned files, and
2. corruption to both your database and versioned files.

Case 1: Database corruption, versioned files unaffected

You will need:
1. The last checkpoint file, which should be available from the latest P4ROOT directory backup.

2. The current journal file, which should be on a separate filesystem from your P4ROOT directory, and which should therefore have been unaffected by any damage to the filesystem where your P4ROOT directory was held.

You will not need:
1. Your backup of your versioned files; if they weren’t affected by the crash, they’re already up to date.

To recover the database

1. Stop the current instance of p4d:

p4 admin stop

(You must be a Perforce superuser to use p4 admin.)

2. Rename (or move) the database (“db.*”) files:

mv your_root_dir/db.* /tmp

There can be no db.* files in the $P4ROOT directory when you start recovery from a checkpoint. Although the old db.* files are never used during recovery, it’s good practice not to delete them until you’re certain your restoration was successful.

3. Invoke p4d with the -jr (journal-restore) flag, specifying your most recent checkpoint and current journal. If you explicitly specify the server root ($P4ROOT), the -r $P4ROOT argument must precede the -jr flag:

p4d -r $P4ROOT -jr checkpoint_file journal_file

This recovers the database as it existed when the last checkpoint was taken, and then apply the changes recorded in the journal file since the checkpoint was taken.

Case 2: Both database and versioned files lost or damaged

You will need:
1. The last checkpoint file, which should be available from the latest P4ROOT directory backup.
2. Your versioned files, which should be available from the latest P4ROOT directory backup.

You will not need:

Your current journal file. The journal contains a record of changes to the metadata and versioned files that occurred between the last backup and the crash; because you’ll be restoring a set of versioned files from a backup taken before that crash, the checkpoint alone contains the metadata useful for the recovery, and the information in the journal is of limited or no use.

To recover the database

1. Stop the current instance of p4d:

p4 admin stop
(You must be a Perforce superuser to use p4 admin.)

2. Rename (or move) the corrupt database (“db.”) files:
mv your_root_dir/db.* /tmp

The corrupt db.* files aren’t actually used in the restoration process, but it’s safe practice not to delete them until you’re certain your restoration was successful.

3. Invoke p4d with the -jr (journal-restore) flag, specifying only your most recent checkpoint:

p4d -r $P4ROOT -jr checkpoint_file

Some Tips While Setting up Perforce Server Configuration to have best Backup & Recovery methodlogy

1. Make sure you are using Symbolic Links for storing versioned files in another drive
2. Make sure you are Storing Journal files in differnet directory
2. Make sure you are storing check points in different directory

Tagged : / / /

How to Backup Perforce Server

How to Backup Perforce Server

Regular backups of your Perforce data are vital. The key concepts are:

  • Make sure journaling is active
  • Create checkpoints regularly, and
  • Use p4 verify regularly
  • Versioned files

checkpoint is a snapshot or copy of the database at a particular moment in time.

journal is a log that records updates made to the database since the last snapshot was taken.

How to Make sure journaling is active

For UNIX installations, journaling is also automatically enabled. If P4JOURNAL is left unset (and no location is specified on the command line), the default location for the journal is $P4ROOT/journal.

How to create a checkpoints

To create a checkpoint, invoke the p4d program with the -jc (journal-create) flag:

p4d -r root -jc

You can create a checkpoint while the Perforce server (p4d) is running. The checkpoint is created in your server root directory (P4ROOT).

To make the checkpoint, p4d locks the database and then dumps its contents to a file named checkpoint.n in the P4ROOT directory, where n is a sequence number.

To create a checkpoint without being logged in to the machine running the Perforce server, use the command:

p4 admin checkpoint [-z] [prefix]

Versioned files

Your checkpoint and journal files are used to reconstruct the Perforce database files only. Your versioned files are stored in directories under the Perforce server root, and must be backed up separately.

Backup Procedures

1. Verify the integrity of your server and add MD5 digests and file length metadata to any new files:

p4 verify //…

You may wish to pass the -q (quiet) option to p4 verify. If called with the -q option, p4 verify produces output only when errors are detected.

2. Make a checkpoint by invoking p4d with the -jc (journal-create) flag, or by using the p4 admin command. Use one of:

p4d -jc
or:
p4 admin checkpoint

3. Ensure that the checkpoint has been created successfully before backing up any files. (After a disk crash, the last thing you want to discover is that the checkpoints you’ve been backing up for the past three weeks were incomplete!) You can tell that the checkpoint command has completed successfully by examining the error code returned from p4d -jc, or by observing the truncation of the current journal file.

4. Once the checkpoint has been created successfully, back up the checkpoint file, the old journal file, and your versioned files.

Tips:

  • Use separate physical drives for server root and journal
  • Use protections and passwords: Until you define a Perforce superuser, every Perforce user is a Perforce superuser, and can run any Perforce command on any file. After starting a new Perforce server, use:> p4 protect and To set (or reset) a user’s password, use p4 passwd username (as a Perforce superuser),
  • Allocate disk space for anticipated growth
Tagged : / / /

Perforce Server Installation and Configuration

p4d – Perforce Server
p4 – Perforce Client
p4win – Perforce Windows Client

Install P4 Server in Unix

  • Copy p4d & p4 in /usr/local/bin
  • Give permission such as chmod +x p4 & chmod +xp4d
  • Create Perforce ROOT directory such as P4ROOT=/usr/local/p4_directory
  • Give permission to P4 root directory and sub directory
  • Tell perforce server to use which port such as P4PORT=servername:1666 or p4d -p 1666
  • Startting Server with Command Line Setup. p4d -r /usr/local/p4root -J /var/log/journal -L /var/log/p4err -p 1666 &
  • Tellin perforce client to which port to listen such as p4_server_hostname:1666
  • Stopping Server e.g. p4 admin stop
Tagged : / / /

Linux User Management

ac Print statistics about users’ connect time.
accton Turn on accounting of processes. To turn it on type “accton /var/log/pacct”.
adduser Ex: adduser mark – Effect: Adds a user to the system named mark
chage Used to change the time the user’s password will expire.
chfn Change the user full name field finger information
chgrp Changes the group ownership of files.
chown Change the owner of file(s ) to another user.
chpasswd Update password file in batch.
chroot Run command or interactive shell with special root directory.
chsh Change the login shell.
edquota Used to edit user or group quotas. This program uses the vi editor to edit the quota.user and quota.group files. If the environment variable EDITOR is set to emacs, the emacs editor will be used. Type “export EDITOR=emacs” to set that variable.
faillog Examine faillog and set login failure limits.
finger See what users are running on a system.
gpasswd Administer the /etc/group file.
groupadd Create a new group.
grpck Verify the integrity of group files.
grpconv Creates /etc/gshadow from the file /etc/group which converts to shadow passwords.
grpunconv Uses the files /etc/passwd and /etc/shadow to create /etc/passwd, then deletes /etc/shadow which converts from shadow passwords.
groupdel Delete a group.
groupmod Modify a group.
groups Print the groups a user is in
id Print real and effective user id and group ids.
last Display the last users logged on and how long.
lastb Shows failed login attempts. This command requires the file /var/log/btmp to exist in order to work. Type “touch /var/log/btmp” to begin logging to this file.
lastcomm Display information about previous commands in reverse order. Works only if process accounting is on.
lastlog Formats and prints the contents of the last login.
logname Print user’s login name.
newgrp Lets a suer log in to a new group.
newusers Update and create newusers in batch.
passwd Set a user’s pass word.
pwck Verify integrity of password files.
pwconv Convert to and from shadow passwords and groups.
quota Display users’ limits and current disk usage.
quotaoff Turns system quotas off.
quotaon Turns system quotas on.
quotacheck Used to check a filesystem for usage, and update the quota.user file.
repquota Lists a summary of quota information on filesystems.
sa Generates a summary of information about users’ processes that are stored in the /var/log/pacct file.
smbclient Works similar to an ftp client enabling the user to transfer files to and from a windows based computer.
smbmount Allows a shared directory on a windows machine to be mounted on the Linux machine.
smbpasswd Program to change users passwords for samba.
su Ex: su mark – Effect: changes the user to mark, If not root will need marks password.
sulogin Single user login.
ulimit A bash builtin command for setting the processes a user can run.
useradd Create a new user or update default new user information.
userdel Delete a user account and related files.
usermod Modify a user account.
users Print the user names of users currently logged in.
utmpdump Used for debugging.
vigr Edit the password or group files.
vipw Edit the password or group files.
w Display users logged in and what they are doing.
wall Send a message to everybody’s terminal.
who Display the users logged in.
whoami Print effective user id.
Tagged : /

Java Command Line Options

-client       to select the “client” VM
-server       to select the “server” VM
-hotspot      is a synonym for the “client” VM  [deprecated]
The default VM is client.

-cp <class search path of directories and zip/jar files>
-classpath <class search path of directories and zip/jar files>
A ; separated list of directories, JAR archives,
and ZIP archives to search for class files.
-D<name>=<value>
set a system property
-verbose[:class|gc|jni]
enable verbose output
-version      print product version and exit
-version:<value>
require the specified version to run
-showversion  print product version and continue
-jre-restrict-search | -jre-no-restrict-search
include/exclude user private JREs in the version search
-? -help      print this help message
-X            print help on non-standard options
-ea[:<packagename>…|:<classname>]
-enableassertions[:<packagename>…|:<classname>]
enable assertions
-da[:<packagename>…|:<classname>]
-disableassertions[:<packagename>…|:<classname>]
disable assertions
-esa | -enablesystemassertions
enable system assertions
-dsa | -disablesystemassertions
disable system assertions
-agentlib:<libname>[=<options>]
load native agent library <libname>, e.g. -agentlib:hprof
see also, -agentlib:jdwp=help and -agentlib:hprof=help
-agentpath:<pathname>[=<options>]
load native agent library by full pathname
-javaagent:<jarpath>[=<options>]
load Java programming language agent, see java.lang.instrument
-splash:<imagepath>
show splash screen with specified image

Tagged : / / /

SVN (Subversion) Branching Strategy

Version control systems play a vital role in the success of a software development team. This blog aims to provide a simple branching and merging strategy to manage the code in a fast paced team development environment.

When do you need branching?

You may need to adopt branching in many different scenarios. Let us assume that on a day to day basis you use agile methodologies such as extreme programming or scrum and you have finally released a reasonably complex project with Version 1.0.0.0. Couple of days after the release, you get a request for a new feature that must go as Version 1.1.0.0. You envision it will take couple of weeks to get that feature coded, tested and released. Meanwhile, simple modifications and patches are requested on a day to day basis in the Version 1.0.0.0.

Essentially, with the changing business needs you are required to do a deployment every other day or so i.e. your deployment cycle is 2 days or less.

With a team of 10 developers it will be a nightmare in the absence of a proper branching strategy.

Trunk, branch, and tag are the 3 main divisions in subversion repository.

Trunk:
The code in trunk is always maintained as close to release state as possible. Any developer making changes to the trunk must be absolutely certain that his or her part can be coded, tested and is ready to deploy within 2 days (can vary depending on your length of deployment cycle). If it takes more than 2 days, they are not allowed to directly change the code in the trunk. They have to create a branch.

Branch:
Developer Branch:
Each developer must create his or her own branch if their code will take more time to program than your normal deployment cycle. It is the responsibility of the developer to regularly (timeframe depends on your development environment) merge changes from trunk to his branch.

Feature branch:
Create a feature branch if 2 or more developers are going to work on a new feature that will take considerable time to complete. It is the responsibility of the team lead to merge changes from trunk to this feature branch on a regular basis.

It is always beneficial to merge changes from trunk frequently. Because, after a couple of days conflict between trunk version and branch version can get out of hand and merging will practically be impossible.

When the developer branch or feature branch is ready for release, merge changes back from the branch to trunk.

Tags:
Tag is similar to branch. When you create a branch simply rename the folder from branch/branches to tag. I always use Tags to tag a release whenever a new release version is deployed. This will come in handy if a very minor patch has to be made on the release version when your trunk is temporarily dirty. Ideally, you do not merge from trunk to the tag.

Finally, remember that there is no ideal branching and merging strategy. It pretty much depends on your unique development environment. If you have any other strategy, I would like to hear about that.

It is an exciting time for Subversion as its adoption continues at a dizzying pace in enterprises. I’m out there helping that adoption so I’m a bit late in posting the questions and answers I promised around the three basic branching strategies that I covered in the last two webinars in which I presented (Branching and Merging Strategies for Subversion 1.5 and Advanced Merge Tracking and Branching with Subversion 1.5). Hopefully these will be useful to many of you.

First, a quick reminder on what the three branching strategies were:

1. The unstable trunk which mimics the way that Subversion itself is developed. In this model, development is done on the trunk and a release branch is created around the time of feature completion with the formal promotion process carried out on that branch.

image

 

2. The stable trunk where development is done on system version branches and the promotion process is also conducted on that branch. The trunk is the branch point where the production releases are merged in and parallel development efforts are branched out.

image

3. The agile release strategy where development is done on individual feature branches and a release branch is created late in the process with the feature branches merged to it that will define that release. The formal promotion process is conducted on the release branch and the production version merged to the trunk as well as to all remaining active feature branches.

So on to the questions, first general branching questions:

image

Does the number of branches affect performance of the Subversion server?

No, branches are not just cheap in space and fast to create, they are alo just additional paths not unlike directories in the directory structure of your project. Of course, every tool has scalability issues at some point and Subversion logically has its limits, but in practical use, a scalability issue for Subversion around branches has not yet been experienced.

Can we use multiple strategies on one project?

Absolutely, there are many times when you may find a need to mix at least two branching models on the same project. For example, Internet applications may have very frequent releases focused primarily to address defects along with longer term release efforts to address broader feature development. The frequent releases may follow the unstable branch approach where they are worked primarily on the trunk and are branched, sometimes weekly or even more frequently, to do the formal QA to get to the release. The longer running enhancement releases may use the stable trunk philosophy though obviously the trunk isn’t actually stable, but the rest of the strategy is followed.

What strategy is best for a repository with multiple projects?

There is no need to have a single strategy in such a case. Instead one should use the appropriate method for each of the projects. Since each project should have its own top level directory for its branching model, there is no reason to force a particular method on all projects.

How does continuous integration affect branching strategies?

I’d hope that this isn’t the deciding factor in determining an appropriate branching strategy, but certainly the feasibility of continuous integration is impacted by the strategy chosen. With the unstable branch, continuous integration would be focused on the trunk and thus is stable as well as very useful as it identifies issues in the daily development commits. This is still relatively true when you look at the stable branch strategy where there may be multiple system version branches under development at the same time, but each is the focus of the majority of the development commits. The agile release model is the least accessible for this approach because of the number of active branches to try and monitor. That doesn’t mean continuous integration can’t be used but just that it is more difficult to implement and maintain.

Unstable Trunk questions:

Can this model handle parallel development if it only happens occasionally?

Any model can be morphed to handle exceptional situations, but this isn’t recommended. It is important to use a strategy consistently so that everyone understands where to find things during development and historically. That said, we’ve already talked about the potential of using multiple strategies on the same project so that’s certainly a possibility where the parallel development need could be addressed like with the stable trunk strategy.

Do I ever recommend merging from the trunk to the branch?

Never say never, but the situation would be pretty rare. I could potentially envision where a nearly complete bug fix or even an enhancement of importance is completed quickly enough on the trunk that there could be a compelling reason to merge it into the unreleased, but branched version.

How would you handle branches for customers that will have ongoing customizations for individual customers in this model?

Branches for this purpose can really be applied to any of the three basic branches we’ve discussed. The concept is that the base product continues to evolve and the branching strategy addresses that evolution. As production releases are made, the new revisions are merged to the customer-specific branches which contain customizations for that customer. There is also the potential of customer customizations being merged into the base product. The customer branches support the unique nature of customer specific changes while allowing for an upgrade path and the potential of contributing back to the base product.

Stable Trunk questions:

Version 1.0.1 didn’t get merged back into the trunk, shouldn’t it have been merged back?

Possibly, but I actually didn’t do so on purpose. With all the models, there is the potential that multiple releases are deployed in production and thus need to be supported. Therefore, there is the potential that bug fix releases are being made for multiple releases and merging all of them back to the trunk could mean intertwined versus serial releases. I therefore usually suggest that the actual production releases may be the only ones merged. Obviously, in my diagram that situation didn’t occur so it is a fair question.

There is a merge to the trunk outside of a release point, doesn’t that violate the stable trunk idea?

Yes, that can violate the stable trunk concept, but stability isn’t necessarily limited to just production releases. In this case, the purpose is to have a more up-to-date starting point for the system version branch. How strict you implement the idea of stability is up to you.

Should the development team develop the first version in the trunk or in a release branch?

Since my base principle is keeping it simple, I tend to say do the development on the trunk since there is no reason for creating a branch. The assumption is that parallel development won’t happen until the first release is made and other work won’t be merged until the release. This isn’t being pure to the strategy, but true to the base principle.

Agile Release questions:

In this strategy, is there any attempt to keep the task branches in sync with each other?

No, there really isn’t any way to define how you would keep isolated tasks in sync since there is no idea what will be released together until we get to the definition of the release branch. There is a need to keep the branches sync’d with production releases, but no way to do more.

Is there any reason that an Internet application should NOT be using the agile release method?

Sure, there are still many such applications that are released in a waterfall manner and are unquestionably successful. For example, release agility may not be required if releases are being made frequently enough. Keep in mind that there is more to keep in sync in this model which has to be balanced against any benefits.

Where is the release package built from?

I think the package would be built from the release branch since that’s logically where the promotion model process is conducted, but it can certainly be argued that the testing can be done on the release branch and then the approved revision merged to be tagged from the trunk. My absolute expectation is that you be consistent where you do build and tag.

How can a CM manager make sure all task branches were merged to the release branch?

Today, I would suggest the use of svnmerge.py where merge tracking is recorded in a property for all branch merges. Obviously, this will be easier with 1.5 where the log command can expose this information. Another approach would come from integrating issue management with Subversion where the merges would be reflected in the record for the release.

I’m sure I’ll return to answer some more questions around these branching models and merge tracking before long. Right now, I’m off to help another enterprise successfully implement Subversion.

 

Reference:

http://www.codelathe.com/blog/index.php/2009/07/02/a-svn-branching-strategy-that-works/

http://stackoverflow.com/questions/34975/branching-strategies

http://blogs.open.collab.net/svn/2007/11/branching-strat.html

http://www.javaworld.com/javaworld/jw-01-2008/jw-01-svnmerging.html

http://www.codinghorror.com/blog/2007/10/software-branching-and-parallel-universes.html

http://www.cmcrossroads.com/forums?func=view&catid=75&id=78871

Tagged : / / / /

How to Resolve Windows Installer Problem

Installation of a program means inserting that particular program in your computer so that it can be executed properly. Some of the software programs can be simply copied to the computer and executed without doing anything further; they don’t require any kind of installation process. Many programs come with an executable suite, which requires to be installed. Installation is the process where you will have to unpack some files, copy them to desired locations, tailor the software to suite your hardware and give the desired information to the operating system.

Installation often means that once a program is installed, the user can run the program over and over again, without reinstalling it again before using each time. Until one does not uninstall the program or the program does not allow further execution, you will have to install it again. However, sometimes one can encounter problems while installing a program. Here are some simple steps that will help you in resolving windows installer problems:

teps:

  • First thing to do for resolving windows installer problem is identifying it. When you are trying to install or uninstall something, you might get a warning message like:

“The windows installer cannot be accessed”

“Windows installer service cannot be started”

“Could not start the windows installer service on Local computer. Error 5:

access is denied.”

  • These error messages will often appear on the screen when the installation of the MSI package has failed or when the windows installer service is disabled.
  • Method 1: first unregister windows installer, and then you will have to register it again. For resolving windows installer problems of such a nature, just do the following things. Go on the ’start’ menu and click on the ‘run’ option. In the dialog box start typing ‘msiexec/unreg’, and then press the enter key.
  • Go on the ’start’ menu again and click on the ‘run’ option. In the dialog box start typing, ‘msiexec/regserver’ and then press the ‘enter’ key.
  • Method 2: you can upgrade the windows installer to a higher version or a newer version. For this, open the internet explorer page, and go to the Microsoft website. Go to the link, http://msdn.microsoft.com/downloads. On the left side you will get an option of ’setup and system administration’ and now click on the ’setup’ option.
  • Select the ‘windows installer’, and then choose the appropriate link for your operating system. Now click on the ‘download’ option and install the new version or install the higher version of windows installer.
  • Method 3: for resolving windows installer problem, you might have to uninstall the failed product with the help of an installer cleanup. The description for the windows installer cleanup utility is, http://support.microsoft.com/default.aspx?scid=kb;en-us;290301
  • Method 4: if the windows service is disabled on your computer, then go to the ’start’ menu, select ‘run’ option and type ’services.msc’ and click enter. Now, double click on the option of windows installer.
  • Method 5: check the DCOM and the permission by the system through http://support.microsoft.com/?id=319624
  • Method 6: another thing you can do for resolving windows installer problem is
Tagged : / / / / / / /

Writing a Custom Plugin for Maven

Introduction

In this article, we will learn about Maven which is a project management framework that provides a configurable approach for managing software projects. Maven covers all the necessary phases that happen right from project creation, building, documentation, reporting, installation and deployment. This article begins with the basics of Maven along with the concepts like Project Object Model (aka POM), the various life-cycles in Maven etc. Then it continues with using Maven for creating a project till installation of the project in a local repository. The latter part of the article provides details about creating custom Maven plugins that can be executed in stand-alone mode as well as part of some Maven life-cycle.

Maven Custom Plug-In (Example Code)

Maven Concepts

In this section, we will see the basics of Maven concepts like Project Object Model, the standard lifecycles available in Maven as well as the dependency scope that comes up while dealing with dependency with projects.

Project Object Model

Project information, its dependencies and the artifacts that can be built are described in POM which is essentially a configuration file in XML. POM basically contains instructions on what needs to be done during the various life-cycles that happen during a build. Note that the POM is not coupled to java projects, in fact the project can be written in any language like C, C# etc. For example, consider the sample minimal POM,

 

<project>

    <modelVersion>4.0.0</modelVersion>

    <name>adder</name>
    <url>http://www.javabeat.net/adder</url>
    <groupId>net.javabeat.adder</groupId>
    <artifactId>adder</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

Have a look at the above POM. It defines the information about the project and its dependant projects. We will look into more details about the various POM elements.

 

  • modelVersion – This defines the version of the POM.
  • name – The name of the project.
  • groupId – This identifies the group (or the organization) that created the project. This property usually takes the domain address which represents the group or the organization so that uniqueness is maintained across the groups.
  • artifactId – This identifies the artifact (output component) name that comes from a project. Artifacts can be anything like a JAR, WAR, EAR etc
  • version – This property identifies the version of the artifact. Usually the version takes the format as “majorVersion.minorVersion-qualifier”. For example, “1.1-alpha”, “1.2-beta”, “1.3-SNAPSHOT”. packaging – Represents the type of artifact, for example – jar, war, ear etc.

In the next section in the POM, we have included the dependencies of the project. Note that this section is optional. If there are no dependencies for a project (which is unlikely the case), this can be omitted. However, in this example case, we have declared a dependency to junit and it is expected that the project will contain some test resources. It is always a good idea to define the version of the artifact while defining a dependency, so that during runtime, we are not facing any issues because of version match. We have mentioned the scope as test, more details about this element will follow in the forthcoming section.

Dependency Scope

A project can depend on a number of dependent projects and Maven provides a way for defining the dependencies among projects through configuration. One can control the level of dependency through scope and the default being ‘compile’. The following scopes are available.

 

  • Compile – This being the default scope will be available at the classpath during the compilation as well as during runtime.
  • Runtime – The presence of this scope ensures that the dependency will be available at the runtime and not during the compile-time. For example, one may use the Servlet API during the compilation phase, however the Servlet implementation (from Sun, Oracle, IBM etc) will be required during the runtime only.
  • Test – This scope ensures that the dependency is available during the compilation of the test resources as well as during the execution of the test resources.
  • Provided – This dependency is mostly used while developing J2EE applications where the dependency is available as part of the J2EE container (or any Container) and it is not necessary for the project to package the dependency as part of the artifact. For example, while developing Enterprise bean applications, as part of the project, we will be using EJB jar for compilation; however, we never package this jar as part of the project because the EJB container will provide this jar.

Build Life-cycle

life-cycle defines the execution for achieving a set of goals. Each life-cycle defines a sequence of phases for achieving the goals. The

Maven Custom Plug-In (Example Code)

Using Maven

In this section, we will see how to use maven for creating, compiling, building, packaging and installing a project. Download the latest version of Maven from here. Installation of maven will be as simple as un-zipping the downloaded file into a desired directory. In this example, we will be seeing the usage of various maven plugins and goals that contributes to the entire project life-cycle. Before proceeding with the sample, create an environment variable MAVEN_HOME that points to the Maven installation directory and ensure that the PATH variable contains %MAVEN_HOME/bin.

Creating a project

Execute the following command which is used to create the initial project structure.

 

mvn archetype:create -DgroupId=net.javabeat.emailservice -DartifactId=emailservice

Note that in archetype:createarchetype represents the plugin prefix and create represents the goal. This goalcreates a project with the name emailservice. As soon as this command is executed, one can see a directory called emailservice which has the src folder for holding the main java files, along with java files representing test-cases. The POM file with the name pom.file will also be created with the structure as follows,

pom.xml

<project>
    <modelVersion>4.0.0</modelVersion>

    <groupId>net.javabeat.emailservice</groupId>
    <artifactId>emailservice</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>emailservice</name>
    <url>http://maven.apache.org</url>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

Importing the project in Eclipse

This step is optional for those who are not working with Eclipse being the IDE for developing the project. Change the current directory to emailservice and issue the following command. This goal eclipse creates the project metadata files that are complaint with Eclipse. As soon as the following goal is executed there will be two files generated in the current directory which are .project and .classpath. This is for Eclipse to understand so that the newly created project can be imported in Eclipse IDE to work with.

 

mvn eclipse:eclipse

Now, for importing the project into Eclipse, Open Eclipse, go to File -> Import -> Existing Projects into Workspace and navigate to the directory containing the .project and .classpath files and Click OK. Now this project gets imported into Eclipse workspace.

Adding files to project

To make the project meaningful, we will add the following files into the main source folder.

EMail.java

package net.javabeat.emailservice;

public class EMail {

    private String fromAddress;
    private String toAddress;
    private String subject;
    private String message;
    private String status;

    public String getFromAddress() {
        return fromAddress;
    }

    public void setFromAddress(String fromAddress) {
        this.fromAddress = fromAddress;
    }

    public String getToAddress() {
        return toAddress;
    }

    public void setToAddress(String toAddress) {
        this.toAddress = toAddress;
    }

    public String getSubject() {
        return subject;
    }

    public void setSubject(String subject) {
        this.subject = subject;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }		
}

The above class represents the Email model object that will be used by EmailService class for sending email. Given below is the code listing for EMailService class.

EMailService.java

package net.javabeat.emailservice;

public class EMailService {

    public EMail sendEmail(String fromAddress, String toAddress, String subject, String message){

        EMail emailObject = new EMail();
        emailObject.setFromAddress(fromAddress);
        emailObject.setToAddress(toAddress);
        emailObject.setSubject(subject);
        emailObject.setMessage(message);
        emailObject.setStatus("SUCCESS");

        System.out.println("Sending email from " + fromAddress + " to " + toAddress + "with subject "
		    + subject + " having the message contents " + message);
        return emailObject;
    }

In this section we will see how to write custom maven plugins. It is possible that the custom maven can be made to run outside the standard life-cycle phase or it can be made to be executed as part of the standard life-cycle phase. We will see how to accomplish both of these items in the following sections.

Creating standalone plugins

In this section, we will write a custom plugin and will see how to write it in stand-alone mode – i.e running the plugin directly without establishing any dependencies on any of the standard life-cycles available in Maven.

Creating the project

The first step is to create the project containing the custom maven plugin. This project is quite different from the regular java project because it is not a regular java artifact like JAR, WAR etc. Instead it is going to be a maven plugin. Issue the following command to create the maven plugin project.

 

mvn archetype:create -DgroupId=net.javabeat.maven -DartifactId=environment-info -DarchetypeArtifactId=maven-archetype-mojo

The above command creates a project with the name environment-info with the standard directory layout containing the source and test resources. If you look into the POM file, the package type will be maven-plugin to indicate that this is a maven plugin project.

Importing the project into Eclipse

This step is optional and this is for someone who wishes to work with the project in Eclipse IDE. Issuing the following with the current directory being environment-info creates the Eclipse .project and .classpath files. Now, open Eclipse and go to File -> Import -> Existing Project into Workspace and specify the directory to <<path to environment-info>> directory.

 

mvn eclipse:eclipse

Creating the plugin

In Maven, the plugin that achieves a specific task is called by the name MOJO (Maven Old Java Object similar to POJO). A MOJO defines the goal which is nothing but the custom action which is executed upon user’s request. In this example, we will see a custom MOJO that will define the goal of storing all the environmental properties in a text file upon execution. Have a look at the following code,

EnvironmentInfoTask.java

package net.javabeat.maven; import java.io.BufferedWriter; import java.io.File; import java.io.FileWriter; import java.util.Iterator; import java.util.Map; import org.apache.maven.plugin.AbstractMojo; import org.apache.maven.plugin.MojoExecutionException; import org.apache.maven.plugin.MojoFailureException; /** * @goal env-info */ public class EnvironmentInfoTask extends AbstractMojo{ /** * @parameter * default-value="log.txt" * expression="${environment.filename}" */ private String fileName; /** * @parameter * default-value="C:\\temp" * expression="${environment.base_dir}" */ private File baseDirectory; /** * @parameter * default-value="true" * expression="${environment.loggingRequired}" */ private boolean loggingRequired; public void execute() throws MojoExecutionException, MojoFailureException { StringBuilder fileContents = new StringBuilder(); if (baseDirectory.exists()){ baseDirectory.mkdirs(); log("Created base directory '" + baseDirectory.getAbsolutePath() + "'"); } Map<String, String> environment = System.getenv(); Iterator<Map.Entry<String, String>> entries = environment.entrySet().iterator(); fileContents.append("Environment Information:" + newLine()); fileContents.append("-----------------------------------------------------------" + newLine()); while (entries.hasNext()){ Map.Entry<String, String> entry = entries.next(); fileContents.append(entry.getKey() + "------------->" + entry.getValue() + newLine()); } fileContents.append("-----------------------------------------------------------" + newLine()); writeToFile(fileContents); log("File contents written"); } private void log(String message){ if (loggingRequired){ System.out.println(message); } } private static String newLine(){ return System.getProperty("line.separator"); } private void writeToFile(StringBuilder fileContents){ FileWriter fWriter = null; BufferedWriter bWriter = null; try{ fWriter = new FileWriter(baseDirectory + File.separator + fileName); bWriter = new BufferedWriter(fWriter); bWriter.write(fileContents.toString()); }catch (Exception e){ log("Error in writing the contents to file ->" + e.getMessage()); }finally{ try{ if (bWriter != null){ bWriter.close(); } if (fWriter != null){ fWriter.close(); } }catch (Exception e){ log("Error in closing the resources ->" + e.getMessage()); } } } }

The first thing to notice is the class EnvironmentInfoTask which is going to persist the environmental information into a file extends AbstractMojo class which in turn extends the Mojo interface. Note that the goal for this MOJO is annotated using XDoclet using goal attribute with the name env-info. The method that will be called during execution of the MOJO is execute(). This method does the job of persisting the environmental properties by taking information from the various properties .

 

  • Base Directory – the directory under which the log file name will be created which is of type ‘java.io.File’. Note that we have used XDoclet ‘parameter’ for annotating this property. This property has the default value ‘C:\\temp’ which means the this is the default location for the log file. However, this value can be overridden at runtime by specifying the property ‘environment.base_dir’ which is an expression and its value will be resolved at runtime
  • Filename – the name of the log file into which the environmental properties will be written. Note that the default value for the log file name is ‘log.txt’. However, during runtime, the value can be overridden through the property ‘environment.filename’
  • Logging Required – whether logging information on what this MOJO is doing has to be shown. The default value being true can be overridden by specifying the value false to the property ‘environment.loggingRequired’.

Compiling the project

Issue the following command for compiling the maven plugin project.

 

mvn compile

Packaging the project

The following command is used to package the plugin project.

 

mvn package

Installing the project

For installing the project into the local Maven repository, issue the following command.

 

mvn install

Usually, on Windows, the local repository is %USER_HOME%/.m2/repository. On Windows XP, USER_HOME will expand to "C:\\Document and Settings\\<<UserName>>", however in Windows Vista it is "C:\\Users\\<<UserName>>".

Running the project

There are several ways to run the maven plugin. We will see them one by one. Generally running a Maven plugin takes the following form,

 

mvn groupId:artifactId:version:goalName

In our case, the group id is net.javabeat.maven, artifact id is environment-info, version is 1.0-SNAPSHOT and goal name is env-info. So running the following command will run the custom plugin

 

mvn net.javabeat.maven:environment-info:1.0-SNAPSHOT:env-info

Running the above command will create a directory C:\\temp (if not present) and create a file name with log.txt with the environment information. Because logging is enabled by default, the text ‘Created base directory C:\temp‘ and ‘File contents are written’ will be displayed in the console.

It is also possible to configure the plugin by passing alternate input values. Have a look at the following command. This command specifies an alternate base directory C:\\temp-new for the log file newLog.txt with logging turned off. Note that for passing a property to a plugin the standard way is –DpropertyName=propertyValue.

 

mvn net.javabeat.maven:environment-info:1.0-SNAPSHOT:env-info -Denvironment.base_dir=C:\temp-new -Denvironment.filename=newLog.txt -Denvironment.loggingRequired=false

It is also possible to omit the version information while running the plugin in which case, Maven tries to find the recent version for the plugin by comparing the version strings. In our case, this wouldn’t happen, because we have only one version which is 1.0-SNAPSHOT. So the following command will also work.

 

mvn net.javabeat.maven:environment-info:env-info

Another approach to run the plugin is by omitting the group id and just specifying the artifact id along with the goal name. However, for this to happen, we have to add the group id to the default list of groups that Maven will look for. Open the file %MAVEN_HOME%/conf/settings.xml and add the following line under the element pluginsGroup.

 

<pluginGroup>net.javabeat.maven</pluginGroup>

This ensures that we have added the group net.javabeat.maven to the list of default groups that Maven will search for, because of which the following command will work.

 

mvn environment-info:env-info

Attaching plugins to existing life-cycle

So far, we have seen how to run a custom plugin in stand-alone mode. This is not of much-use and Maven provides a way for binding the goals of any custom plugin into existing build cycle. For example, let us consider that we have written a custom plugin which will simply print the current date and we want this plugin to be executed during the compilation phase and the installation phase of a project. We will see how to accomplish this goal in this section.

The following listing shows the custom plugin which will simply emit the date information.

DateInfo.java

package net.javabeat.maven; import java.util.Date; import org.apache.maven.plugin.AbstractMojo; import org.apache.maven.plugin.MojoExecutionException; import org.apache.maven.plugin.MojoFailureException; /** * @goal date-info */ public class DateInfo extends AbstractMojo { public void execute() throws MojoExecutionException, MojoFailureException { Date currentDate = new Date(); getLog().info("Current date is " + currentDate); } }

Follow the same process of creating the project, then creating the plugin, compilation, packaging and installation as we have discussed before. We will see how to attach this custom plugin during the compile and the install phase. Open POM.xml file and add the following after the project dependencies element,

 

<build> <plugins> <plugin> <groupId>net.javabeat.maven</groupId> <artifactId>environment-info</artifactId> <version>1.0-SNAPSHOT</version> <executions> <execution> <id>date-info-compile</id> <phase>compile</phase> <goals> <goal>date-info</goal> </goals> </execution> <execution> <id>date-info-install</id> <phase>install</phase> <goals> <goal>date-info</goal> </goals> </execution> </executions> </plugin> </plugins> </build>

Though the above XML fragment is big, it is fairly simple. First thing is, it defines the execution element where we can define goals for executions. We have defined two execution elements one for the compile phase and the other for the install phase and we have given appropriate id for each execution element. Have a note at the goals section. We have included our custom goal date-info which will display the current date information. Note that where to search for the goal’s group id and the artifact id along with optional version is defined just above the executions element.

After this whenever we run mvn compile and mvn install for any project that contains the above definition we will see that the goal date-info getting invoked during the phases.

Conclusion

In this article, we have seen how to use Maven for creating projects and to manage dependencies between projects. We have seen Maven simplifying the task of creating, compiling, packaging, installing projects thereby reducing the pain from developers and build managers. Also discussed in the article are the extension capabilities provided by Maven for pluging-in custom plugins in the form of MOJOs.

}

Note that there is no real functionality in the above, all it does is to accept the various user inputs and constructs an email object before returning it.

Compiling the project

The next step is to compile to project, by default the compilation process will create a folder called target and it will place the java class files along with any resource files, if present.

 

mvn compile

Note that the location of the target folder as well as the location of the target folder is also configurable.

Packaging the project

This step will package the project with the packaging type mentioned in POM, which is JAR. Before packaging it will also execute the test cases (if specified) in the test folder and the packaged jar will be placed in the target folder.

 

mvn package

Installing the project

The final step is to install the jar in a local repository so that other projects, if they are dependant on this project can make references to this project. The following command will install (i.e copy the jar) to the local repository.

 

mvn install

Note that, by default, the local repository path will be %USER_HOME%/.m2/repository.

Maven Custom Plug-In (Example Code)

Tagged : / / / / /

Syncing missing/modified files to a specific version in Perforce

I think this is the 5th time I’ve had to figure this out in the past few years, so this time I’m going to preserve it here for posterity :)

The problem is usually one or more of:

  • You have files missing from your workspace that need replaced.
  • You have locally modified (i.e. not opened for edit) files that need replaced.
  • These files are large in either size or number, and on a slow or remote server, so a complete force-sync is not an appealing option.

Normally this could easily be solved with something like p4 diff -sd/-se //depot/path/.. | p4 -x – sync -f, which finds all the missing/outdated files and then force-syncs them.

The problem with this approach comes when you need the files from a specific revision or label. While you can pass in a label/revision to ‘p4 diff’, the sync part will always retrieve the latest version of a file. To fix this we need to save the list of files we need to a temporary location, then postfix each line with the version we want, then feed that new list back to p4 sync.

But there’s one final catch. If your filenames are local paths instead of Perforce paths, you cannot specify a label, only revisions. E.g. c:\temp\foo.dat@SOME_LABEL will not work, but c:\temp\foo.dat#999 will. Fortunately we can use the pseudo revision ‘have’ for our purposes.

@echo off

REM Could optionally pass these as %1 and %2 from the cmd line
set LABEL=@SOME_LABEL
set BASE_DIR=//depot/path/…

del p4sync.txt

echo Setting ‘have’ to %LABEL% for %BASE_DIR%
REM do a nop sync that sets our ‘have’ version to the label we want
p4 sync -k %BASE_DIR%%LABEL%

echo Finding modified files..
p4 diff -se %BASE_DIR%%LABEL% > p4diff.txt

echo Finding missing files..
p4 diff -sd %BASE_DIR%%LABEL% >> p4diff.txt

echo Building sync commands
REM Append #have to each filename and put them all in a list of files to sync
for /f %%i in (p4diff.txt) Do echo %%i#have>> p4sync.txt

REM Now sync everything
if exist p4sync.txt (
echo Syncing..
p4 -x p4sync.txt sync -f
)else (
echo It seems no files need updated!
)

Voila!

Tagged : / / / / / /

Customizing the Firefox Installer on Windows

One of the questions I get asked a lot is how to customize the Firefox installer on Windows and how to bundle extensions with it. I’ve spent the past few days learning a great deal about this subject, so I thought I would take this opportunity to provide a refresher on working with the Firefox installer on Windows. I’m going to do it as a Q&A so hopefully folks will get answers to the common questions they have.

Standard disclaimer: Under no circumstances should you use this information to create a custom Firefox install and redistribute it to anyone outside your organization. If you want more information, you can consult the Mozilla Foundation Trademark Policy.

What tools do I need to work with the Firefox installer?

The primary tool you need is 7-Zip. I install the MozillaBuild package which gives me all the tools I need. Even though the Firefox Installer is NSIS based, we will not need to use NSIS for most customizations. I’ll talk a little bit about the end about what kinds of things you would need NSIS to do.

How do I unpack the Firefox installer?

The Firefox installer is created using 7-Zip. So you can grab any of the Windows installers that end in EXE and unpack them. Any of the Windows installers on the Firefox download page will work. Once you’ve downloaded the EXE, create a temporary directory and type:

7z x "Firefox Setup 3.6.3.exe"

This will unpack the contents of the installer so we can modify it.

How do I bundle my extension with the Firefox installer?

Bundling your extension with the Firefox installer is just a matter of putting it in the right place. Then when we package up the installer at the end, it will get installed along with Firefox. For most extensions, the right place is nonlocalized/extensions. Inside that directory, create a subdirectory that corresponds to the ID of the extension you want to preinstall with Firefox. Then unzip the XPI into that directory. You can find the ID by looking at the install.rdf file inside the XPI. You can add as many extensions as you want into the installer.

What are some useful extensions I can bundle with Firefox

I’ve created two extensions that create interesting things to bundle with Firefox. The first is the CCK Wizard. The CCK Wizard can be used to change various defaults in Firefox so that you can customize it for deployment in your organization. The second is Rebrand. Rebrand allows you to change the internal branding used in Firefox.

Can I change the names used in the installer?

Yes, you can change the names used in the installer. To do this, you need to create a directory called distribution inside the localized directory that was created when you unpacked the installer. Create a file called setup.ini in this directory. Here’s what it looks like:

[Branding]
BrandFullName=Mike's Browser
BrandShortName=Browser

BrandFullName will be used to replace “Mozilla Firefox” and BrandShortName will be used to replace “Firefox”.

Can I change the images used in the installer?

Yes, you can change the images used in the installer. In that same directory where you put the setup.ini, you can put two files, modern-wizard.bmp and modern-header.bmp. The first images corresponds to the large image on the first page of the installer. The second image corresponds to the small image that is used on later pages of the installer. You can use the linked images as a reference to know what size to make these images.

How do I repackage the installer?

To repackage the installer, first you need to zip up the changes that you made. Type:

7z a -r -t7z app.7z -mx -m0=BCJ2 -m1=LZMA:d24 -m2=LZMA:d19 -m3=LZMA:d19 -mb0:1 -mb0s1:2 -mb0s2:3

This will create a file called app.7z that has all the changes we made. Now we need to package that file with some other files to create the final EXE. We’ll need the file 7zSD.sfx which you can download from Mozilla. And we’ll need a file called app.tag which you can create. It looks like this:

;!@Install@!UTF-8!
Title="Mozilla Firefox"
RunProgram="setup.exe"
;!@InstallEnd@!

Once we have these files, we can run the command:

copy /B 7zSD.sfx+app.tag+app.7z our_new_installer.exe

to package them all as an EXE. Don’t forget the /B. It indicates that the files are binary so Windows won’t put an EOF marker on them.

Can I change the defaults that are set in the installer like the install directory or the checkboxes?

At this time, there is no way to change the defaults in the installer without rebuilding the installer. There’s a bug open on this with a patch, so hopefully this will be fixed for Firefox 4.

Can I make my totally rebranded Firefox coexist nicely with an existing Firefox?

There are a couple ways to do this. The easiest way is to use the -no-remote parameter when you start Firefox. This causes the Firefox you are starting to not connect to the Firefox that is currently running. When you do this, you have to specify a different profile using the -P parameter. Alternatively, you can change the internal identifiers that Firefox uses. Then it will be considered to be a completely different browser. If you choose to do this, you should be aware that you will not receive updates and there will be other side effects. This is not a decision that should be taken lightly. Also, your profiles will be stored in different locations as well. If you want to do this, check out the file application.ini in the nonlocalized directory. The variables you want to change are Vendor and Name. Again, you do this at your own risk.

What can I do if I’m willing to rebuild the installer with NSIS?

If you are willing to rebuild the installer, you can change things like the name of the entry in the Add/Remove programs list, as well as the install directory and other defaults. This is a nontrivial exercise because some of the required files are built as part of the Mozilla build proces and are not available in the build tree. If you’re really interested in doing this, you can contact Kaply Consulting and we can talk about it.

I hope this answered some questions folks have. If anyone has any more questions, please don’t hesistate to ask.

Tagged : / / / / /