Work on remote Subversion repositories locally with Git

remote-subversion-git

Work on remote Subversion repositories locally with Git

Version control is great stuff, and being able to combine different version control mechanisms is even better. Subversion is a very popular version control system and a lot of repositories (public or otherwise) use Subversion to manage files. Git is another popular one, but what happens if you are working on a project where Subversion is used but Git is your preferred version control system?

With the git-svn plugin, you can have the best of both worlds. You can convert a Subversion repository to Git, use Git tools, then push the changes back to Subversion.

To begin, you will need the git-svn plugin installed. Most likely, if your distribution of choice provides Git, it will also provide git-svn. On Fedora, install it using:

# yum install git-svn

Then use git-svn to check out your Subversion repository into Git format:

% mkdir -p ~/git/code

$ cd ~/git/code

% git svn init http://svn.host.com/svn/code

Initialized empty Git repository in /home/user/git/code/.git/

% git svn fetch

This may take a while on large repositories

r267 = 079b7c1cff49187d1aabc4b16f316102088fdc0d (refs/remotes/git-svn)

W: +empty_dir: trunk

r268 = 3f1944530a092c811c55720bd9322b8c150a535b (refs/remotes/git-svn)

r351 = e2af3c12e5ed174d23ffc5917f03a6136f8ebb6b (refs/remotes/git-svn)

Checked out HEAD:

http://svn.host.com/svn/code r351

At this point, the Subversion repository located at http://svn.host.com/svn/code has now been checked out in Git format. On individual files and directories, you can use the git log command as you would the svn log command in order to get history information on the item in question. With git, you will also see the Subversion commit that corresponds to the log entry, for instance:

commit 23f705cd87e1e9c6dd841ca88a14d808e0c4292a

Author: user@HOST.COM

Date:   Sat Mar 20 18:25:38 2010 +0000

correct logic on the buildrequires extractor, add stats on BuildRequires to showdbstats output

git-svn-id: http://svn.host.com/svn/code@350 7a5473d1-2304-0410-9229-96f37a904fa4

With the above, you can see that user@HOST.COM did the commit, see the log message, and the Subversion revision (r350).

To work with these files, make changes as normal. git diff works like svn diff does, to see the changes made. To commit changes, use git commit like you would use svn:

% git commit -m “some minor change” file

[master 2454be1] some minor change

1 files changed, 2 insertions(+), 0 deletions(-)

To update your local copy from Subversion, instead of using svn update use git svn rebase. This will merge in any changes found in the Subversion repository.

When committing files using git commit, you are committing your changes to the Git repository. None of these changes are pushed to the Subversion repository until you specifically tell Git to do so. This is done with the git svn dcommit command, which then takes each commit made to Git and pushes them to Subversion as individual commits, which will retain all of your history and log comments:

% git svn dcommit

Committing to http://svn.host.com/svn/code …

M      trunk/rqp

Committed r352

M      trunk/rqp

r352 = 0557314a580c4390ff646380baa3aa33d1f6a5cd (refs/remotes/git-svn)

No changes between current HEAD and refs/remotes/git-svn

Resetting to the latest refs/remotes/git-svn

Unstaged changes after reset:

M      trunk/rqp

M      trunk/rqp

Committed r353

M      trunk/rqp

r353 = 249e97283ad28126bf84ccaffb32873e12d15b7b (refs/remotes/git-svn)

No changes between current HEAD and refs/remotes/git-svn

Resetting to the latest refs/remotes/git-svn

Now, if you were to look at the changed file(s) in Subversion (via another Subversion working copy or something like ViewVC), you will see the individual commits. Above, there were two changes made to the trunk/rqp file, each committed locally to Git. The “dcommit” command pushed those changes as individual commits to the Subversion repository. In this way you can do all local development with Git and when you have something you want to commit to the Subversion repository, you can push all of the relevant changes at once, retaining each separate commit.

Using the git-svn plugin makes it extremely easy to use Git locally with a remote Subversion repository. If you are in a project or organization that, for whatever reason, does not want to convert to Git, you can continue to work with that Subversion repository, without the restriction of using Subversion yourself.

Tagged : / / / / / / / / / / / / / / / / / / /

How to configure and use SSH authentication system server CVS

ssh-with-cvs

How to configure and use SSH authentication system server CVS

cvs (Concurrent Version System) is a very popular version control tool. Although its function as Perforce, Subversion and other powerful, but because of its easy configuration, simple to use and the introduction of longer, so in all a kind of software project widely used.

first is to install FreeBSD, and application security patches. This step is very simple, download a FreeBSD (the upcoming 5.2-RELEASE is a good choice, in this article was published, this version may have been able to download a) of the mini iso burning CD-ROM, then you can install the (choose “Minimum” installation). Note that, in order to be able to use ssh authentication, be sure to install the crypto. Configure the network for this machine, so after the restart just fine.

then install the necessary package. FreeBSD 4.x built-in perl, but in 5.x in, perl be deleted from the basic system. In order to better use, including ACL, commit mail, and a number feature, you must install perl.

I personally recommend using the ports in the 5.8.x version of perl. Before installing, first of all convinced that the system does not perl 5.6.x installed, the new system, they can safely execute the following command:

This will remove the existing any package. In general, the majority of FreeBSD users use cvsup to update the system, but the release of the CD, cvsup-without-gui is not included, while the ordinary cvsup package requires a series of X11 libraries. To avoid trouble, consider the following two different ways to complete the first code update:

1, using the FreeBSD’s cvs to synchronize code:

cvs-d: pserver: anoncvs@anoncvs.jp.FreeBSD.org: / home / ncvs login

enter “anoncvs”, Enter

cd / usr

cvs-R-d: pserver: anoncvs@anoncvs.jp.FreeBSD.org: / home / ncvs export-r RELENG_5_2 src ports

Note: If you are not using 5.2-RELEASE, please make appropriate amendments to RELENG_5_2 (for example ,4-STABLE is RELENG_4, – CURRENT is the HEAD, etc.)

2, install cvsup-without-gui:

pkg_add-r cvsup-without-gui

then use cvsup to synchronize code, space is limited, I will not repeat them

I recommend the first method, of course, the second method should be faster, and does not require manually compiling cvsup. Subsequently, the installation of Perl 5.8.x:

cd / usr / ports / lang / perl5.8 & & make all install clean & & rehash

Then we will face a very serious problem: ports inside use.perl script does not know the system had not installed perl, so the time to be wrong in the implementation – this time, just make a symbolic link to perl on it. Well, the implementation:

use.perl port

This script will help you complete use.perl modify configuration files (such as / etc / make.conf) and a series of work. Here is not the table, a little profile about OpenSSH (sshd) configuration, in general, add the following two lines:

Protocol 2

PasswordAuthentication no

the benefits of doing so: ( 1) only allow ssh2 protocol log, which can provide better security (2) does not allow password login, which no doubt will increase security.

1, create a cvs repository

Well, pre-configuration is basically over. Do not forget to create a user for the cvs group, for example, ncvs, as well as a cvs repository for the management of users, such as the repoman (which of course is ncvs that group), and then create a directory to save the cvs repository, in this case , we put it into / home / ncvs in:

rm-rf / home / ncvs

mkdir-p / home / ncvs

chown-R repoman: ncvs / home / ncvs

chmod-R 775 / home / ncvs

next step is to initialize the repository, and simple to implement:

su-l repoman

cvs-d / home / ncvs init

on it.

present, all the FreeBSD version of cvs are included in the safety of the existence of a small vulnerability, although this flaw only in local use, but we recommend that you put it back on is to find / usr / src / contrib / cvs / src / expand_path.c in

return current_parsed_root-> original;

line, turning it into a

return current_parsed_root-> directory;

Of course, the next step is to re-make world kernel of. Need to note is that if you are using a 5-CURRENT, also need to modify some code for it to reach 5.2-RELEASE as performance (-CURRENT in a large number of debugging options turned on), specific methods will not go into here.

2, the configuration commitmail and ACL

then configure cvs commitmail and ACL. I personally think that commitmail the team software development a very important thing to cvs, it is especially important because cvs is not atomic submit function, but commitmail just enough to make up for this.

FreeBSD development team used a very good perl script to complete commitmail function, while the hook they use cvs to achieve a simple access control function (ACL). I use the cvs repository based on FreeBSD’s CVSROOT, and made a few changes.

use the CVSROOT cover your CVSROOT, and you also need to freebsd directory mailsend.c which compiled the results into the / usr / local / bin (the script assumes that the CVSROOT in this matter). In addition, the CVSROOT need some modifications before it is put into use (for example, machine name, etc.). These settings can be found in the cfg_local.pm inside:

$ MAILADDRS = ‘cvs- all@example.org ‘;

commitmail this is the place to be sent to.

$ MAIL_BRANCH_HDR = “X-Phantasm-CVS-Branch”;

this is to be added to the commitmail head of information, if you use the mailing list, then it can help mailing list automatically sorted.

$ MAILBANNER = “The Phantasm Studio repository”;

this soon in commitmail said that they commit to which a repository.

if ($ hostname = ~ / ^ cvs \. Example \. Org $ / i)

commit to this is to determine the host name

$ CVSWEB_URL = “http://cvsweb.example.org/cgi-bin/cvsweb.cgi”;

This is cvsweb services at

a brief CVSROOT in the other documents

avail: the file used to control access to user group.

access: This file is used to control who can perform cvs operations.

exclude: This file is used to control those files do not need to check the cvs tag

options: this file used to control the commencement of cvs tag, for example, can define $ Phantasm $, etc.

3, configuration of users and restrict ssh permissions

ssh authentication using a more vexing question is, ssh means that users have a system account, and that they be able to log in.. If configured properly, they can get a shell, it is natural to be a potential security risk.

must therefore be very careful in handling the ssh cvs authentication issue. The underlying principle is: any action against the user, unless we allow them to do so.

create a user in accordance with the following rules:

user’s” primary “group is ncvs (This will not only restrict the user’s permission, so we can more easily control the other users in the cvs repository which can not be free to commit)

users do not use password authentication, which will ease because they do not properly set up ftp and other safety hazards caused by

still give the user a shell, the “shell” can be a perl script, it only allow the implementation of the beginning of the command cvs

then allow users to use openssh’s ssh-keygen to generate your own key pair. OpenSSH in most * BSD and Linux distributions can be found in, if the user uses the Windows Desktop, you need to install cygwin (in particular, the installation of net in OpenSSH), of course, the implementation of the command is the same:

ssh-keygen-t dsa-b 2048

for the paranoid security enthusiasts, can consider later in 2048 will be replaced by-b 4096. Of course, under the barrel theory, if you are a paranoid, then obviously you should be forced to use all the partners are also at least as long as your key:)

, the administrator should be user (committer) the key into the server, the user corresponding to the authorized_keys file in the folder inside. For example, one of my public key as follows:

ssh-dss AAAAB3NzaC1kc3MAAAEBAL +1 jinOw +86 RcTEaSM5/Hz4Lr9tIS0IQsX8ebo

TwLzWnqpOHRh2KBCGn/e0xGCIAai7PGz7c + SZCvrLiRvG9mCsMMMue8ZIL + QF4OAmMd

Cz8Qoyg0cc4YXImOd + UEpdOX29PC4aMAz28v/GO2yf58/Qa49Clfq1kHa/8q3IAgs9o

W95 / ArG + IWFOsN1Tv9nh4XJb5AQjpa5uMlB5SEmvKGTXQ2oYiRVIxL8vzHL6MtO/8×1

j8 + RioSH6FCpEXS7UJbYxE7vF3m5Fa5o6g2dIZewphsleOeHkvYJ442Hqvsly3p4 +4 N

dvim4bY2HMDha5r5zeTV8tTlOz4wQVgKyWoEAAAAVAINGzX7uU0vR8l63qhBhUeWGZt

C9AAABADWiO +9 bvV7DApsn08LR1eoEnMjJFQgEfGlbV + EvZHkO0bkHZAdRIKtVmgNUw

G6uufykkt2Tb + q5SbVNZkzeaFVv4ZMtnjSvEPIZrEXcQFFguGk1it5v5EYcmq4G8 + j1

BFTVHef4b1wMTSt11WtEz0LUYncuZ6LA48/WGTuZiSm8JkchgVm8HhR9NqjdeFJH8sO

RUhUBoxyWjo/hv7zFg7HqoJGzeNfrEhFg36psR2RDaRvSP0vN1W2q4j5OZy3gB6ZyVt

nsEPl1HELhlaCFifmdz1LVxDx + FyPy6wMsPQLTmB1g6N1J6PWy3qCTJ0NyQgarSt3 / A

TQ0InF1BOdJn8QAAAEAPb1OgswuMHdEsHk2ETZVmOKOkI9Rjf72vjZ3xG45iEbCH/7p

aTP8OQmJMW9FD4MHjdmtktPVYXDIa9Hj/IM44zhfMHEdKs9LlFUK5dBgNUps + yPj2Ns

Mr2rl771ODR0mB52FwrXm1FCmNTM7WQpFOEy/QhtZRpHK +7 / YZp7PBggt17Fw7rbjP2

zhWnZluoSKLgvfkhxhJuOMm/ElNJx2c + XHdxPqI3eR5UxzLNjDUNh59I8 + h + E69bFB3

b2uhKqziziHOQcqoH5r0Kud / DBBE79lU3mRUF8FQNygCRh/V3yFzed40rc0nF0PQpNZ

6zodDTJByrm6vX5wr2lI4RgA9w == bitripper@grimreaper.delphij.net

Description: public key is not allowed off-line, here is convenient for typesetting. We have just opened to the user shell, which is still potential pitfalls, so we tighten security in this regard, the public key by adding the following text before the project:

command = “/ usr / bin / cvs – allow-root = / home / ncvs server”

So, the whole line should look like this:

command =” / usr / bin / cvs – allow-root = / home / ncvs server “ssh-dss AAAAB

…… …………………..

X5wr2lI4RgA9w == bitripper@grimreaper.delphij.net

ssh this command means that up on the implementation of / usr / bin / cvs – allow-root = / home / ncvs server this command, and can only perform this command. Thus, unless there are loopholes in the cvs itself, otherwise, can not really cut out through ssh shell. – Allow-root limits the ability to use the repository, so users can not easily specify the other repository, thus trying to undermine the security will become more difficult. If you need multiple repository, you can specify the number of allow-root parameter.

be noted is that if users have multiple public key, you need to specify each public key before the command.

4, configuration cvsweb

configuration cvsweb is quite simple. First install the apache (I tried 1.3.x and 2.0.x), configure the ready, from the port which install cvsweb on it:

cd / usr / ports / devel / cvsweb & & make all installl clean

and then to modify it configuration file:

cd / usr / local / etc / cvsweb

vi cvsweb.conf

find

;

@ CVSrepositories = (

line, follow the example of an increase in following their own repository, For example:

‘ncvs’ => [‘ New CVS Repository ‘,’ / home / ncvs’],

write the number of how many, of course, @ CVSrepositories the first to directly access cvsweb.cgi when the repository.

in FreeBSD 5.1-CURRENT development process carried out by some of the changes caused by the ports in the cvsweb not normally call the cvs and rcs tools, which will cause it to not work properly. To solve this problem, can be found from the following web site the latest version of cvsweb:

http://www.freebsd.org/projects/cvsweb.html

the author of this writing, the latest cvsweb version is 2.9.1 beta. Before the installation, need to install the other two port:

cd / usr/ports/devel/p5-IPC-Run & & make all install clean

cd / usr/ports/net/p5-URI & & make all install clean

then downloaded tbz file unpack, copy to the appropriate location (cgi-bin and related images directory) can be used.

Tagged : / / / / / / / / / / / / / / /

Upgrading Continuum – Continuum Upgradation Guide

continuum-upgrade

This document will help you upgrade Continuum from 1.2.x to 1.3.3 and above.

When upgrading Continuum, it could have some database model changes. Usually these changes will be migrated for you, but in some cases you may need to use a backup from the previous version and restore that data into the new version. The Data Management tool exports data from the old database model and imports the data into the new database model.

If you had used the APP_BASE environment variable in Continuum 1.2 to differentiate your configuration from the installation, you should rename it to CONTINUUM_BASE in Continuum 1.3.

Note: The Jetty version in Continuum 1.3.4 and above has been upgraded to 6.1.19. When upgrading to Continuum 1.3.4 or higher, there is a need to update the library contents listed in $CONTINUUM_BASE/conf/wrapper.conf with the ones included in the new distribution especially if the $CONTINUUM_BASE directory is separate from the installation.

Using Backup and Restore to upgrade

There are 2 databases that need to be considered: one for the builds and one for the users.

There were no changes in the users database from 1.2.x to 1.3.2, so you can simply point Continuum 1.3.2 at your existing user database.

The builds database has had model changes, and will need to be exported and imported.

First, download the Data Management tools you will need. The tool is a standalone JAR that you can download from the central repo.

You will need to download two versions of the tool, one for the export out of the old version and one for the import into the new version:

Note: The 1.2, 1.2.2 and 1.2.3 released versions of this tool have a bug. To export databases from 1.2.2 or 1.2.3, you will need to use version 1.2.3.1 of the tool. To export databases from 1.2, you may use the 1.1 version of the tool.

Next, follow these steps to export data from the old version

  • Stop the old version of Continuum
  • Execute this command to create the builds.xml export file
java -Xmx512m -jar data-management-cli-1.2.x-app.jar -buildsJdbcUrl jdbc:derby:${old.continuum.home}/data/databases/continuum -mode EXPORT -directory backups

Then, follow these steps to import the data to the new version

  • Start the new version of Continuum to create the new data model, but do not configure it.
  • Stop Continuum
  • Execute this command to import the builds data from the xml file you created earlier:
java -Xmx512m -jar data-management-cli-1.3.2-app.jar -buildsJdbcUrl jdbc:derby:${new.continuum.home}/data/databases/continuum -mode IMPORT -directory backups -strict

Note: Remove -strict when importing data from 1.3.1 to 1.3.x to ignore unrecognized tags due to model changes.

Finally, be aware that sometimes the NEXT_VAL values in the SEQUENCE_TABLE need to be adjusted.

  • Before starting Continuum for the first time after the import, connect to the db with a client like Squirrel SQL and check the values in the NEXT_VAL column of the SEQUENCE_TABLE.
  • Values must be greater than the max id value in each table.
  • For example, the next value of “org.apache.maven.continuum.model.Project” must be greater than the greatest id in Project table.
  • Here are some example SQL statements. You may need to add or remove lines depending on the contents of your database.
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(systemconfiguration_id)+1 from SYSTEMCONFIGURATION) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.system.SystemConfiguration';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from BUILDQUEUE) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.BuildQueue';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from SCHEDULE) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.Schedule';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from BUILDDEFINITION) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.BuildDefinition';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from LOCALREPOSITORY) WHERE SEQUENCE_NAME='org.apache.continuum.model.repository.LocalRepository';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from PROJECTGROUP) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.ProjectGroup';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(scmresult_id)+1 from SCMRESULT) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.scm.ScmResult';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(projectdependency_id)+1 from PROJECTDEPENDENCY) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.ProjectDependency';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from BUILDDEFINITIONTEMPLATE) WHERE SEQUENCE_NAME='org.apache.maven.continuum.model.project.BuildDefinitionTemplate';
UPDATE SEQUENCE_TABLE set NEXT_VAL = (select max(id)+1 from ABSTRACTPURGECONFIGURATION) WHERE SEQUENCE_NAME='org.apache.continuum.model.repository.AbstractPurgeConfiguration';

Now you can start your new version of Continuum.

Reference: http://continuum.apache.org/docs/1.3.6/installation/upgrade.html

 

 

Tagged : / / / / / / / / / / / / / / /

How to Run/Deploy Java EE applications on Amazon EC2?

running-java-ee-applications-on-amazon-ec2

Running Java EE applications on Amazon EC2: deploying to 20 machines with no money down

Computer hardware has traditionally been a scarce, expensive resource. In the early days of computing developers had to share a single machine. Today each developer usually has their own machine but it’s rare for a developer to have more than one. This means that running performance tests often involves scavenging for machines.  Likewise, replicating even just part of a production environment is a major undertaking. With Amazon’s Elastic Compute Cloud (EC2), however, things are very different. A set of Linux servers is now just a web service call away. Depending on the type of the servers you simply pay 10-80 cents per server per hour for up to 20 servers! No more upfront costs or waiting for machines to be purchased and configured.

To make it easier for enterprise Java developers to use EC2, I have created EC2Deploy.  It’s a Groovy framework for deploying an enterprise Java application on a set of Amazon EC2 servers. EC2Deploy provides a simple, easy to use API for launching a set of EC2 instances; configuring MySQL, Apache and one or more Tomcat servers; and deploying one or more web applications. In addition, it can also run JMeter and collect performance metrics.

Here is an example script that launches some EC2 instances; configures MySQL with one slave, Tomcat and Apache; deploys a single web application on the Tomcat server; and runs a JMeter test with first one thread and then two.

class ClusterTest extends GroovyTestCase {
  void testSomething() {
    AWSProperties awsProperties = new
        AWSProperties("/…/aws.properties")

    def ec2 = new EC2(awsProperties)

    def explodedWar = '…/projecttrack/webapp/target/ptrack'

    ClusterSpec clusterSpec =
       new ClusterSpec()
            .schema("ptrack", ["ptrack": "ptrack"],
                    ["src/test/resources/testdml1.sql",
                     "src/test/resources/testdml2.sql"])
            .slaves(1)
            .tomcats(1)
            .webApp(explodedWar, "ptrack")
            .catalinaOptsBuilder({builder, databasePrivateDnsName ->
                 builder.arg("-Xmx500m")
                 builder.prop("com.sun.management.jmxremote")
                 builder.prop("com.sun.management.jmxremote.port", 8091)
                 builder.prop("com.sun.management.jmxremote.authenticate",
                                     false)
                 builder.prop("com.sun.management.jmxremote.ssl", false)
                 builder.prop("ptrack.application.environment", "ec2")
                 builder.prop("log4j.configuration",
                               "log4j-minimal.properties")
                 builder.prop("jdbc.db.server", databasePrivateDnsName)})

    SimpleCluster cluster = new SimpleCluster(ec2, clusterSpec)

    cluster.loadTest("…/projecttrack/functionalTests/jmeter/SimpleTest.jmx",
        [1, 2])

    cluster.stop()
  }
}

Let’s look at each of the pieces.

First, we need to configure the framework as follows:

    AWSProperties awsProperties = new
        AWSProperties("/…/aws.properties")
    def ec2 = new EC2(awsProperties)

The aws.properties file contains various properties including the Amazon WS security credentials and the EC2 AMI (i.e. OS image) to launch. All servers use my EC2 appliance AMI that has Java, MySQL, Apache, Tomcat, Jmeter and some other useful tools pre-installed.

Next we need to configure the servers:

     ClusterSpec clusterSpec =
        new ClusterSpec()
             .schema("ptrack", ["ptrack": "ptrack"],
                    ["src/test/resources/testdml1.sql",
                     "src/test/resources/testdml2.sql"])
             .slaves(1)
             .tomcats(1)
             .webApp(explodedWar, "ptrack")
             .catalinaOptsBuilder({builder, databasePrivateDnsName ->
                 builder.arg("-Xmx500m")
                 builder.prop("com.sun.management.jmxremote")
                 builder.prop("com.sun.management.jmxremote.port", 8091)
                 builder.prop("com.sun.management.jmxremote.authenticate",
                                     false)
                 builder.prop("com.sun.management.jmxremote.ssl", false)
                 builder.prop("ptrack.application.environment", "ec2")
                 builder.prop("log4j.configuration",
                               "log4j-minimal.properties")
                 builder.prop("jdbc.db.server", databasePrivateDnsName)})

     SimpleCluster cluster = new SimpleCluster(ec2, clusterSpec)

This code first creates a ClusterSpec, which defines the configuration of the machines and the applications:

  • schema() – specifies the name of the database schema to create; names of the users and their passwords; the DML scripts to execute once the database has been create
  • slaves() – specifies how many MySql slaves to create
  • tomcats() – specifies how many Tomcats to run.
  • webApp() – configures a web application. This method takes two parameters: the path to the exploded WAR directory (conveniently created by Maven) and the context to deploy the web application under.
  • catalinaOptsBuilder() – supplies a closure that takes a builder and the DNS name of the MySQL server as arguments and returns the CATALINA_OPTS used to launch Tomcat. It’s primary purpose is to configure the web application(s) to use the correct database server

It then creates a cluster with that specification.

We then start the cluster:

    cluster.start()

At this point EC2Deploy will:

  1. Launch the EC2 instances running my appliance AMI.
  2. Initialize the MySql master database
  3. Create the MySql slave
  4. Create the database schema and the users
  5. Run any DML scripts (these are cached on S3 in a bucket called “tmp–dml” for the reasons described next)
  6. Upload the web applications to Amazon S3 (Simple Storage Service) where they are cached in order to avoid time consuming uploads (over slow DSL connections, for example). EC2Deploy only uploads new and changed files, which means that the bulky 3rd party libraries are only uploaded once. Each web application is stored in an S3 bucket called -tmp-war. If this bucket does not exist you will see some warning messages and the bucket will be created.
  7. Deploy the web applications on each of the Tomcat servers
  8. Configure Apache to load balance across the Tomcat servers

Once the cluster is started we can run a JMeter load test:

    cluster.loadTest("…/projecttrack/functionalTests/jmeter/SimpleTest.jmx", [1, 2])

The first argument specifies the test to run and the second argument is a list of JMeter thread counts. In this example, EC2deploy first runs the load test with one thread and then two threads. For each test run, it generates a report describing CPU utilization for each machine, average response time and throughput.

Finally, we stop the EC2 instances:

cluster.stop()

As you can see, EC2Deploy makes it pretty easy to deploy and test your enterprise Java application. I’ve used it to clone a production environment and run load tests. NOTE 1/28/08: The source code EC2Deploy along with a very cool Maven plugin is now available !

Tagged : / / / / / / / / / / / / / / / / / /

Running MSBuild 4.0 and MSBuild 3.5 on Continuous Integration

msbuild-40-and-msbuild-35

With Visual Studio 2010 RC released recently, we jumped on the release and began to code with VS2010.  One issue that popped up was that now all builds were targeting MSBuild 4.0.

That doesn’t seem to be a big problem until our CruiseControl CI server kicked in, downloaded our updated code and failed building the upgraded projects.

Fortunately there is a very quick solution to this little problem.  There are a couple of requirements.

1. You need to have VS2010 RC installed somewhere
2. You need to download the .Net Framework 4.0 (I recommend the full version and not just the Client Profile, it ensures you don’t miss anything)

To fix, do the following:

1. download and install the .Net Framework 4.0 on the CI server (then restart the server)
2. on the computer where VS2010 RC is installed go to the following path:
%programfiles%\MSBuild\Microsoft\VisualStudio
3. copy the v10.0 folder located in that directory into the CI server at the same path (or wherever our MSbuild path is on the CI server)
4. Once that is done, edit the ccnet.config file at the tag and change it to the new .Net 4.0 Framework installed (you should only need to change the section “\v3.5\” to “\v4.0.xxxxx\”

Hope this helps

Tagged : / / / / / / / / / / / / / / /

Issues Compiling VS2010 solutions (with web projects) from Nant | MSB4064 error

vs2010-compiling-issues

Recently I upgraded a project of mine (the Dimecasts code base) to use VisualStudio 2010.  In the process everything worked just fine from the IDE, but when I tried to compile it from the command line I would get the following errors:

Error MSB4064: The “Retries” parameter is not supported by the “Copy” task.
Error MSB4063: THe “Copy” task could be initialized with its input parameter.

After a bit a googling I came across a post which (and of course i cannot find it now) said that if you open up your .proj files and change the line that pointed to the v10.0 build of web applications and reset it back to 9.0 everything would compile.  And this did work… BUT when you try to open that project up again in VS 2010 it will simply revert your changes… this is not a working solution.

Next I decided to switch my target framework in Nant from 3.5 to 4.0, but of course my nant.exe.config file does not support 4.0 yet.  So after a bit more googling I found this post that gives details on how to add the missing values to the config file.

When I added the config information to my Nant.exe.config file things were better, but still not great.  Now I was getting an error that said:

The “vendor” attribute does not exist, or has no value.

To resolve this I added the following under the node in my config
vendor=”Microsoft”

After this I got another error…. This time it said that .Net Framework 4.0 was not installed.  But I know this is not valid.  After looking at the information for a few more seconds I realized the issue.  The example config from the post above was build on an older version of the 4.0 framework (.20506) and I have .30128.

I changed all values in the nant.exe.config value that was v4.0.20506 to be v4.0.30128 and NOW I am able to compile.

So long story short, if you are getting the MSB4064 error you need to do the following:

1. Point nant to use the 4.0 framework tools
2. Follow this post  and copy the framework section to your Nant.exe.config file
3. Add the missing ‘vendor’ attribute to the new framework section
4. Update the version in the new framework section to match the version you have on disk (check C:\Windows\Microsoft.NET\Framework for versions)
5. Compile again

Tagged : / / / / / / / / / / / / / / / / / / /

Why and how to use Jetty in mission-critical production

how-to-use-jetty

This article is a summary of a seminar I had on the topic. If it seems like it’s a continuation of an existing discussion that’s because, to some extent, it is. If you haven’t been discussing exchanging your app server, this article probably isn’t very interesting to you.

By putting the application server inside my application instead of the other way around, I was able to leap tall buildings in a single bound.

The embedded application server

This is how I deploy my sample application to a new test environment (or to production):

  1. mvn install
  2. scp -server/target/-1.0.onejar.jar appuser@appserver:/home/appuser/test-env1/
  3. ssh appuser@appserver “cd /home/appuser/test-env1/ && java -jar -1.0.onejar.jar&”

This require no prior installed software on the appserver (with the exception of the JVM). It requires no prior configuration. Rolling back is a matter of replacing one jar-file with another. Clustering is a matter of deploying the same application several times.

In order to make this work in a real environment, there are a many details you as a developer need to take care of. As a matter of fact, you will have to take responsibility for your operational environment. The good news is that creating a good operational environment is not more time-consuming than trying to cope with the feed and care of a big-a Application Server.

In this scheme every application comes with its own application server in the form of jetty’s jar-files embedded in the deployed jar-file.

The advantages

Why would you want to do something like this?

  • Independent application: If you’ve ever been told that you can’t use Java 1.5 because that would require an upgrade of the application server. And if we upgrade the application server, that could affect someone else adversely. So we need to start a huge undertaking to find out who could possibly be affected.
  • Developer managed libraries: Similar problems can occur with libraries. Especially those that come with the application server. For example: Oracle OC4J helpfully places a preview version of JPA 1.0 first in your classpath. If you want to use Hibernate with JPA 1.0-FINAL, it will mostly work. Until you try to use a annotation that was changed after the preview version (@Discriminator, for example). The general rule is: If an API comes with your app server, you’re better served by staying away from it. A rather bizarre state of affairs.
  • Deployment, configuration and upgrades: Each version of the application, including all its dependencies is packaged into a single jar-file that can be deployed on several application server, or several times on the same application server (with different ports). The configuration is read from a properties-file in the current working directory. On the minus side, there’s no fancy web UI where you can step through a wizard to deploy the application or change the configuration. On the plus side, there is no fancy web UI …. If you’ve used one such web UI, you know what I mean.
  • Continuous deployment: As your maven-repository will contain stand alone applications, creating a continuous deployment scheme is very easy. In my previous environment, a cron job running wget periodically was all that was needed to connect the dots. Having each server environment PULL the latest version gives a bit more flexibility if you want many test environments. (However, if you’re doing automated PUSH deployment, it’s probably just as practical for you).
  • Same code in test and production: The fact that you can start Jetty inside a plain old JUnit test means that it is ideal for taking your automated tests one step further. However, if you test with Jetty and deploy on a different Application Server, the difference will occasionally trip you. It’s not a big deal. You have to test in the server environment anyway. But why not eliminate the extra source of pain if you can?
  • Licenses: Sure, you can afford to pay a few million $ for an application server. You probably don’t have any better use for that money, anyway, right? However, if you have to pay licenses for each test-server in addition, it will probably mean that you will test less. We don’t want that.
  • Operations: In my experience, operations people don’t like to mess around with the internals of an Application Server. An executable jar file plus a script that can be run with [start|status|stop] may be a much better match.

The missing bits

Taking control of the application server takes away a lot of complex technology. This simplifies and makes a lot of stuff cheaper. It also puts you back in control of the environment. However, it forces you to think about some things that might’ve been solved for you before:

  • Monitoring: The first step of monitoring is simple: Just make sure you write to a log file that is being monitored by your operations department. The second step requires some work: Create a servlet (or a Jetty Handler) that a monitoring tool can ping to check that everything is okay. Taking control of this means that you can improve it: Check if your data sources can connect, if your file share is visible, if that service answers. Maybe add application-calibrated load reporting. Beyond that, Jetty has good JMX support, but I’ve never needed it myself.
  • Load balancing: My setup supports no load balancing or failover out of the box. However, this is normally something that the web server or routers in front of the application server anyway. You might want to look into Jetty’s options for session affinity, if you need that.
  • Security: Jetty supports JAAS, of course. Also: In all the environments I’ve been working with (CA SiteMinder, Sun OpenSSO, Oracle SSO), the SSO server sends the user name of the currently logged in user as an HTTP header. You can get far by just using that.
  • Consistency: If you deploy more than one application as an embedded application server, the file structure used by an application (if any) should be standardized. As should the commands to start and stop the application. And the location of logs. Beyond that, reuse what you like, recreate what you don’t.

Taking control of your destiny

Using an embedded application server means using the application server as a library instead of a framework. It means taking control of your “main” method. There’s a surprisingly small number of things you need to work out yourself. In exchange, you get the control to do many things that are impossible with a big-A Application Server.

Reference: http://www.javaworld.com/community/

Tagged : / / / / / / / / / / / / / / /

How to copy VSS project from one VSS database to another one without loosing history

copy-vss-project-from-one-vss-database-to-another-one-without-loosing-history

If you have any of the following questions in your mind, then this article is the perfect destination for you.

  • How to copy VSS project from one VSS database to another one without loosing history
  • How to move VSS project from one location to another location
  • Moving Visual SourceSafe projects or databases to a new location?
  • Moving a project or projects from one database to another, while preserving the project’s history?

The procedures within each scenario may vary depending on the version of Visual SourceSafe you are running.

APPLIES TO

Microsoft Visual SourceSafe 5.0 Standard Edition

Microsoft Visual SourceSafe 6.0 Standard Edition

NOTE: This process is for Visual SourceSafe 5.0 or greater only.

Requirement:

1.   Use the SSARC and SSRESTOR utilities included in Visual SourceSafe 5.0. With these utilities, you can archive projects, preserve their histories, and restore them to a new database.

2.   You must have SourceSafe Admin privileges to use these utilities

3.   In order to restore a project successfully your archive must include the latest versions all the files in the project. If you use the -v switch with SSARC to archive off earlier versions of your files, you will not be able to restore that archive to a different database because it needs the later versions.

4.   Because the physical file names get renamed when you restore a project to a new database, it may be necessary to reconnect projects that are integrated with Visual SourceSafe.

Warning:

1.   always be very careful when archiving and deleting files

2.   Backing up source files is like voting, do it early and often (warning, this may not be legal in your jurisdiction).

3.   Test your backups out before you actually need them, avoid permanent deletes and keep copies in multiple locations.

 

There are two ways to Archive and Restore Project/Files from one database to another.

To Archive:

1.   Using To archive a database using Visual SourceSafe Administrator

2.   SSARC

Archival of a database allows you to:

* Save disk space on the Visual SourceSafe server or other machine hosting the database.
* Make the Show History command work more quickly.
* Transport files and projects between Visual SourceSafe databases, keeping history information intact.
* Provide a compressed file backing up all or part of a database.

How to Archive Visual SourceSafe (VSS) Project?

Method1: Using Visual SourceSafe Administrator:

The Visual SourceSafe Administrator program allows you to archive your databases using the Archive Projects command, and to restore them from backup using the Restore Projects command.

To archive a database using Visual SourceSafe Administrator:

1. Ensure that no one is using the database and that the ANALYZE utility will not run during the archive operation.
2. Open the database for which you want to archive information.
3. On the Archive menu of Visual SourceSafe Administrator, click Archive Projects.
4. In the Archive wizard, select the project(s) to archive, and click OK.
5. If there are additional projects to be archived, indicate this in step 1 of the wizard, and click Next when finished.
6. In step 2 of the wizard, specify how you want the project(s) to be archived, and click Next when finished. For each choice, you can use Browse to search for and select an archived file to overwrite the existing file.
7. In step 3 of the wizard, specify the versions for the data to archive, using the Version box.
8. Add a comment to the Comment box to document the archive operation.
9. Click Finish to complete the wizard. The archive is written to a *.ssa file in the directory that you have specified.

Method2: Using SSARC: –

Another way of making an archive is to use the Visual SourceSafe SSARC utility from the command line.
Examples

SSARC -d- -yadmin,password archive.ssa $/
Backup the entire default Sourcesafe database to archive.ssa, leaving the database exactly as it is.

SSARC -d- “-vlProduction Release” -yadmin,password -olog.txt archive.ssa $/Test

Backup everything since the version labelled ‘Production Release’ and create a log file with the results of the archiving process.

SSARC -d -x -yadmin,password archive.ssa “$/Project Global Domination” $/OtherProject

Description:  Archive, and delete the deleted files from two projects.

SSARC -i -yadmin,password -olog.txt “-cArchive Everything” archive.ssa $/

Description:  Archive the entire Sourcesafe database, while creating a log file, adding a comment, and running non-interactively, suitable for a scheduled task.
To Restore Project/Files from one database to another.

To Restore:

1.   Using To archive a database using Visual SourceSafe Administrator

2.   SSRESTOR

 

How to Restore Visual SourceSafe(VSS)Project?

Method 1: Using Visual SourceSafe Administrator

Visual SourceSafe Administrator supports restoration of a database from an archive using its Restore Projects command.

To restore a database using Visual SourceSafe Administrator

1. Make sure that no one is using the database and that the ANALYZE utility will not run during the restore operation.
2. On the Archive menu of Visual SourceSafe Administrator, click Restore Projects.
3. In the Restore wizard, enter or locate the archive file from which to restore, and then click Next.
4. In step 2 of the wizard, you can choose the items to restore to the database, and click Next to proceed.
5. Use step 3 of the wizard to specify options for restoring the selected items to the database.
6. Add a comment to the Comment box to document the restore operation.
7. Click Finish to complete the wizard.
8. Verify the database directories carefully to make sure that the restore operation has put the restored items where you intended.

Method 2: Using SSRESTOR Utility

Restores information from a previously created archive. If the restore operation attempts to create a duplicate file or project name, the operation fails. Visual SourcSafe also implements restore operations through its Restore wizard, available in Visual SourceSafe Administrator.
Examples

SSRESTOR -la -yadmin,password archive.ssa $/

Display a list of all of the files archived in archive.ssa.

SSRESTOR “-p$/Test 2” -sD:\newfolder\ -yadmin,password backup.ssa $/Test

Restores project $/Test to the database to a new location, $/Test 2, from the archive file backup.ssa, in a different Sourcesafe database.

Tagged : / / / / / / / / / / / / / / / / / / /

SSARC Utility & SSRESTOR Utility – Archive, Restore VSS Project – Guide

ssarc-utility-ssrestor-utility

Question: 

How to Archive VSS Project in Visual Source Safe (VSS)?

How to Restore VSS Project in  Visual Source Safe (VSS)?

What is SSARC Utility?

What is SSRESTOR Utility?

SSARC Utility

Allows you to archive files, projects, or old versions from a Visual SourceSafe database. Each time you run SSARC (Ssarc.exe), the utility asks only once before it deletes the files/projects. Visual SourceSafe also implements archive operations through its Archive wizard, available in Visual SourceSafe Administrator.

The utility SSRESTOR is used for restoration of archived files. Use of SSARC and SSRESTOR together allows wide-area Visual SourceSafe installations to move files and projects among databases.

Limitation: SSARC cannot create an archive file that is greater than 2 GB. If you try to archive a project larger than this, you will receive an Out of Memory and/or a CRC mismatch error message. To work around this limitation, you will need to archive each subproject.

 

Syntax

ssarc [-C][-D][-I-][-O][-S][-V][-X][-Y] <archive file to create> <files/projects to archive>

If you are not familiar with the syntax conventions used, < and > are used to delimit arguments. The < and > characters are not typed in. The | means a choice of options. Items enclosed in square brackets, [] are optional, and items not enclosed in square brackets are mandatory.

 

Utility Options

The following table describes options available with SSARC.

-C

Specifies a comment (standard Visual SourceSafe parameter). The comment is inserted into the Visual SourceSafe history if the items are deleted from the database as part of the archived record. The comment is also inserted into the archive file itself, if there is one.

-C-

Specifies no comment.

-D

Deletes the archived items from the database.

Specifiying the -d option will delete from your project the items that you are archiving. This will delete your project from Sourcesafe if you use it! Of course, you can restore from your backup file, but for normal use, and for smaller projects I don’t recommend this. This function obviously has its uses, but be sure you want to do this.

-D-

Controls whether Visual SourceSafe actually deletes anything from the database. If this option is not used, the default is to ask.

If the -d- option is used, then SSARC will not delete the projects the items you are archiving. The default is to prompt the user interactively, so if you want to run this from a script, you will need to choose one of the options, or use the -i- flag

-I-

Specifies no prompt for input (standard Visual SourceSafe parameter).

-O

Paginates or redirects output (standard Visual SourceSafe parameter).

-S<srcsafe.ini path>, <data path>

Specifies a path to Srcsafe.ini and the Data directory. The full syntax looks like -Sc: \VSS, OldDB. The item before the comma is the full path to Srcsafe.ini, and the item after the comma is the string in parentheses in a Data_Path setting. If there is no comma, a Srcsafe.ini path is indicated, but no data string. If the first character after the –S is a comma, a data string is specified, but no Srcsafe.ini path.

 

-V[D|L]<version>

Specifies a version number to archive (standard Visual SourceSafe parameter). Enter the version number in standard Visual SourceSafe format (number, date, or label). If this option is not used, SSARC operates on entire files or projects, instead of all versions up to and including a certain version.

Note that SSARC is generally inclusive. That is, if you type –V9, you create an archive file that contains version 9.0 and everything before it. The delete pass is also inclusive, that is, version 9.0 is actually deleted unless it is in the label format. In the latter case, the label is stored in the archive file but is not deleted. If –V is specified, you never delete the current version, even if you specify -D9/9/99. If you are using a version the label format, and the label has a space in it, you must place the entire option in quotation marks, for example, “-VThisOne”.

-X

Archives only deleted items in the specified files and projects. Deleted items are still stored in the Visual SourceSafe database unless the Destroy Permanently option is selected when performing a Delete command.

-X-

Archives all items in the specified files and projects.

-Y<user>,<pwd>

Specifies the user name and password (standard Visual SourceSafe parameter). An example is -YAdmin,Bunny.

archive file to create

Specifies the name of the archive file to create during the archive operation.

files/projects to archive

Specifies files and projects to back up.

Examples

SSARC -d- -yadmin,password archive.ssa $/
Backup the entire default Sourcesafe database to archive.ssa, leaving the database exactly as it is.
SSARC -d- "-vlProduction Release" -yadmin,password -olog.txt archive.ssa $/Test
Backup everything since the version labelled ‘Production Release’ and create a log file with the results of the archiving process.
SSARC -d -x -yadmin,password archive.ssa "$/Project Global Domination" $/OtherProject
Archive, and delete the deleted files from two projects.
SSARC -i -yadmin,password -olog.txt "-cArchive Everything" archive.ssa $/
Archive the entire Sourcesafe database, while creating a log file, adding a comment, and running non-interactively, suitable for a scheduled task.

SSRESTOR Utility

Restores information from a previously created archive. If the restore operation attempts to create a duplicate file or project name, the operation fails. Visual SourcSafe also implements restore operations through its Restore wizard, available in Visual SourceSafe Administrator.

Limitation: SSRESTOR cannot restore a project that is larger than 2 GB. A project larger than this cannot be archived.

 

Syntax:

ssrestor [-C][-I-][-L][-O][-P<project>][-S][-T][-X][-Y]

<archive file to restore> [files/projects]

 

Utility Options

The following table describes options available with this command.

-C

Specifies a comment (standard Visual SourceSafe parameter). The comment is applied to the history entry for restored item(s).

-C-

Specifies no comment.

-I-

Specifies no input (standard Visual SourceSafe parameter).

-L

Specifies a list only, without any restoration.

-LA

Specifies a list of all files and subprojects listed under a project, for example, project $/A.

-O

Redirects output (standard Visual SourceSafe parameter).

-P<project>

Specifies a project to which to restore content. For instance, if you archive $/A/BAR.C and then restore it, it will be restored as $/A/BAR.C.

-S<srcsafe.ini path><data path>

Specifies a path to Srcsafe.ini and the Data directory.

-T

Tests the archive file for corruption, but does not actually restore from the archive.

-X, -X-

Identifies an item that you want to restore, for example, $/a/b. This option distinguishes between deleted projects and undeleted projects that have the same names. For example, if you have deleted $a/b and specify the -X option, SSRESTOR restores the deleted $/a/b. If you specify -X-, SSRESTOR restores an undeleted $/a/b. Even though they have the same name, note that these are two different projects that the utility treats differently.

-Y<user>,<pwd>

Specifies user name and password (standard Visual SourceSafe parameter). An example is -YAdmin,Moggy.

archive file to restore

Specifies the name of the archive file from which to restore the database.

files/projects

Specifies files and projects to restore.

Examples

SSRESTOR -la -yadmin,password archive.ssa $/
Display a list of all of the files archived in archive.ssa.
SSRESTOR "-p$/Test 2" -sD:\newfolder\ -yadmin,password backup.ssa $/Test
Restores project $/Test to the database to a new location, $/Test 2, from the archive file backup.ssa, in a different Sourcesafe database.

Miscellaneous tips

  • Putting the path to the <VSS PATH>\win32 folder in your PATH environment variable makes shell commands a lot easier
  • If you get a message “Only ADMIN can run this utility”, it means that you are either not specifying the admin account, or the password is incorrect. Use "-yadmin,password" if the admin password is “password”, or "-yadmin," if the admin password is not set (not recommended!).
  • If you are using an option that has a space in the argument, enclose the whole thing with quote marks, i.e. “-p$/Project A”
  • A daily archive is easy to do with a Windows shell script, maybe something like this. Then use Scheduled Tasks to have this run every day.
    function q (str)
    	' to make the command a bit more readable, I hope
    	q = """" & str & """"
    end function

    PROJECT = “$/”
    USERID = “admin”
    PASSWORD = “admin”

    ‘ location of SSARC program
    SSARCPATH = “C:\program files\microsoft visual studio\common\vss\win32\ssarc.exe”
    ‘ folder of srcsafe.ini
    SRCSAFEINIPATH = “”
    ‘ prepended to filename in case you’re doing more than one.
    LABEL = “ARCHIVE”
    ‘ destination of archive files
    BACKUPFOLDER = “C:\BACKUPS\”

    ‘ generate a name based on the time.
    today = now()
    backupfilename = LABEL & “-” & formatdatetime(now,2) & “.ssa”

    cmd = q(SSARCPATH) & ” ” & _
    q(“-s” & SRCSAFEINIPATH & “,”) & ” -i- ” & ” -d- ” & _
    q(“-y” & USERID & “,” & PASSWORD) & ” ” & _
    q(BACKUPFOLDER & backupfilename) & ” ” & q(PROJECT)

    Set WshShell = WScript.CreateObject(“WScript.Shell”)
    wscript.echo cmd
    WshShell.run cmd
    set wsshell = nothing

Tagged : / / / / / / / / / / / / / / / / / / / /

How To Create a Unique SRCSAFE.INI File ?

srcsafeini-file
1. Create a new SRCSAFE.INI file and place it in the desired location.
2. If you want include the default SRCSAFE.INI in the unique copy, add the following line:

#INCLUDE <SS path>\SRCSAFE.INI

3. Create the following environment variable:

SET SSDIR=<path to VSS directory>

(such as SET SSDIR = \\MyServer\MyVSSdir)

While it is possible to do this with Visual SourceSafe 5.0, in is not necessary. Simply open the database using file->open.

Tagged : / / / / / / / / / / / / / / /