Why organization should enhance DevOps skills of their employee?

organization-employee-devops-skills

It’s not shocking that a lot of organizations struggle when it comes to DevOps execution. DevOps is a new conception which IT organizations use to enhance collaboration between two functions into one. In today’s cutthroat business environment, DevOps comes as an important resolution that helps organizations meet ever-growing business demands.

 

The meaning of DevOps is different for different peoples. Some says it’s a “collaboration of operations and developers” or it’s using automation or it is a toolchain approach or culture or a job title or it’s just a Dev. and Ops. or variety of related things. But if we look onto it, it’s not just a set of tools or not just a culture or job title. We can say It is a way of doing work where people of both operations and developer skill sets work together on projects to create quality products without wasting resources and infrastructure in a short span of time by using best practices of tools.

 

There are many organizations in the world who are adopting DevOps to enhance their employee Performances and revenues and profitability of their business. There are many studies done till date where it is clearly shown that where companies implement the DevOps approach get more done, plain & simple. The greatest change in approach in a DevOps environment is that there is a single group made out of cross functional members including operations architects, business expert, DBA’s, QA, developers and so forth. There are many advantages of collaboration among these different roles delivers.

 

If we look on to benefits there are:-

1. Continuous software delivery

2. Less complex problems to fix

3. Faster resolution of problems

4. Faster delivery of features

5. More stable operating environments

6. More time available to add value (rather than fix/maintain)

 

To accomplish this, Dev and Ops must break the silos and work together with each other, share liability for keeping up the framework that runs the product, and set up the product to keep running on the framework with continuous quality input and delivery automation.

 

In this process there are so many tools and practices used to achieve the results. Source Code Repository, Build Server, Configuration Management, Virtual Infrastructure, Test Automation and Pipeline Orchestration any many more in the process.

 

DevOps is certainly a group activity, however it’s still vital to clearly define once roles and liabilities for accomplishment inside the group. Means to say it is really important for organizations to make sure dev and ops teams are collaborating and sharing responsibilities throughout the software delivery lifecycle from planning all the way to managing the production environment.

 

Now the real problem is that there are various challenges an organization and their employee’s face when they trying to execute DevOps skills because they do not able to overcome the Developers and Operations mentality and unable to do continuous balance and change of work process, they lack of understanding the security and access management, unable to do continuous monitoring, Toochain cashes of Developers and operations and various others problems they face when they try to implement it on their projects. To overcome these challenges organization’s must provide their employees DevOps skills from industry experts. Now to make your job easy I did some research on it and found one portal “DevOps Consulting” where you can get best industry trainers and consultants for corporate and individual training.

Tagged : / / / / / / / / / / / /

Why Git is Better than X | Git or X

why-git-is-better-than-x

Cheap Local Branching

svn perforce

Everything is Local

bzr svn perforce

Git is Fast

hg bzr svn perforce

The Staging Area

svn perforce

Distributed

svn perforce

Any Workflow

hg svn perforce

GitHub

perforce

Easy to Learn

hg bzr svn perforce

Git is the new standard

Tagged : / / / / / / / / / / / / /

What is Code Coverage and Why Code Coverage?

code-coverage

What is Code Coverage
Code Coverage is an important measurement in Software Quality Engineering. While Software testing ensures correctness of the applications, a metric is required to track the What is Code Coverage Code Coverage is an important measurement in Software Quality Engineering. While Software testing ensures correctness of the applications, a metric is required to track the completeness and effectiveness of the testing undertaken. Code Coverage helps achieve reliable quality through identifying untested areas of the application.

Why Code Coverage
Software testing is a challenging function. The testers need to ensure complete functional and non-functional correctness of the product. Considering the complex workflows and use cases of modern day applications, the number of unique cases that the software can be used often run into millions, which is not feasible to be covered under testing exercise. The testers thus need to
– While Planning Tests
o Ensure covering all workflows in terms of decision trees in the code
o Ensure covering all data values – by identifying patterns rather covering millions of values
– While testing
o Ensuring the testing is completely exercising the whole application with planned and exploratory tests.

At the end of testing, the decision to stop testing and release the product still remains subjective, based on the presence or absence of bugs, inflow of new bugs, success rate of each test cycle, confidence rating of the testers or users, etc. Whereas the definitive metric of quantifying how much of the application was really tested, is missed.

Code Coverage is measured as quantification of application code exercised by the testing activities. Code Coverage can be measured at various levels – in terms of programming language constructs – Packages, Classes, Methods, Branches or in terms of physical artifacts – Folders, Files and Lines. For Eg. A Line Coverage metric of 67% means the testing exercised 67% of all executable statements of the application. A Code Coverage metric usually is accompanied by Code Coverage Analysis Report – which helps identify the un-tested part of the application code, thereby giving the testers early inputs for complete testing.

Benefits of Code Coverage

  • Objective Indicator of Test Coverage of application code
  • Pointers to uncovered Packages / Classes / Methods / Branches
  • Pointers to uncovered Folders / Files / Lines
  • Drill down to untested part of source code and devise new tests
  • Early Indicator for Testing Quality and Fixing it by adding new tests.
  • Remove redundancy in testing
  • Increased Confidence for Releases

Test Your Test

Typical Emotional Storyboard

  • Write Some code! Happy!
  • Does it work? Sad!
  • Write some test! Happy!
  • Do they really test the code? Sad!
  • Measure the Code Coverage! Happy!

Coverage Measurement

  1. Shows Which line of code are executed
  2. How much of your code is covered by your tests?
  3. Your tests test your product
  4. Coverage testing tests your tests

Goal

  • 100%
  • Coverage Ideal
  • Not Always possible
  • Can be expensive to achieve
  • Design for testability

Good: Write more tests
Only way to truly increase code coverage

Bad
Excluding Code to boost Coverage

Types of Coverage

  1. Statement Coverage
  2. Branch Coverage
  3. Path Coverage
  4. Loop Path Coverage
  5. Data – driven Code
  6. Complex Conditionals
  7. Hidden Branches

How; – Coverage Tools

  1. Clover
  2. Cobertura
  3. Emma
Tagged : / / / / / / / / / / / / /

Why and how to use Jetty in mission-critical production

how-to-use-jetty

This article is a summary of a seminar I had on the topic. If it seems like it’s a continuation of an existing discussion that’s because, to some extent, it is. If you haven’t been discussing exchanging your app server, this article probably isn’t very interesting to you.

By putting the application server inside my application instead of the other way around, I was able to leap tall buildings in a single bound.

The embedded application server

This is how I deploy my sample application to a new test environment (or to production):

  1. mvn install
  2. scp -server/target/-1.0.onejar.jar appuser@appserver:/home/appuser/test-env1/
  3. ssh appuser@appserver “cd /home/appuser/test-env1/ && java -jar -1.0.onejar.jar&”

This require no prior installed software on the appserver (with the exception of the JVM). It requires no prior configuration. Rolling back is a matter of replacing one jar-file with another. Clustering is a matter of deploying the same application several times.

In order to make this work in a real environment, there are a many details you as a developer need to take care of. As a matter of fact, you will have to take responsibility for your operational environment. The good news is that creating a good operational environment is not more time-consuming than trying to cope with the feed and care of a big-a Application Server.

In this scheme every application comes with its own application server in the form of jetty’s jar-files embedded in the deployed jar-file.

The advantages

Why would you want to do something like this?

  • Independent application: If you’ve ever been told that you can’t use Java 1.5 because that would require an upgrade of the application server. And if we upgrade the application server, that could affect someone else adversely. So we need to start a huge undertaking to find out who could possibly be affected.
  • Developer managed libraries: Similar problems can occur with libraries. Especially those that come with the application server. For example: Oracle OC4J helpfully places a preview version of JPA 1.0 first in your classpath. If you want to use Hibernate with JPA 1.0-FINAL, it will mostly work. Until you try to use a annotation that was changed after the preview version (@Discriminator, for example). The general rule is: If an API comes with your app server, you’re better served by staying away from it. A rather bizarre state of affairs.
  • Deployment, configuration and upgrades: Each version of the application, including all its dependencies is packaged into a single jar-file that can be deployed on several application server, or several times on the same application server (with different ports). The configuration is read from a properties-file in the current working directory. On the minus side, there’s no fancy web UI where you can step through a wizard to deploy the application or change the configuration. On the plus side, there is no fancy web UI …. If you’ve used one such web UI, you know what I mean.
  • Continuous deployment: As your maven-repository will contain stand alone applications, creating a continuous deployment scheme is very easy. In my previous environment, a cron job running wget periodically was all that was needed to connect the dots. Having each server environment PULL the latest version gives a bit more flexibility if you want many test environments. (However, if you’re doing automated PUSH deployment, it’s probably just as practical for you).
  • Same code in test and production: The fact that you can start Jetty inside a plain old JUnit test means that it is ideal for taking your automated tests one step further. However, if you test with Jetty and deploy on a different Application Server, the difference will occasionally trip you. It’s not a big deal. You have to test in the server environment anyway. But why not eliminate the extra source of pain if you can?
  • Licenses: Sure, you can afford to pay a few million $ for an application server. You probably don’t have any better use for that money, anyway, right? However, if you have to pay licenses for each test-server in addition, it will probably mean that you will test less. We don’t want that.
  • Operations: In my experience, operations people don’t like to mess around with the internals of an Application Server. An executable jar file plus a script that can be run with [start|status|stop] may be a much better match.

The missing bits

Taking control of the application server takes away a lot of complex technology. This simplifies and makes a lot of stuff cheaper. It also puts you back in control of the environment. However, it forces you to think about some things that might’ve been solved for you before:

  • Monitoring: The first step of monitoring is simple: Just make sure you write to a log file that is being monitored by your operations department. The second step requires some work: Create a servlet (or a Jetty Handler) that a monitoring tool can ping to check that everything is okay. Taking control of this means that you can improve it: Check if your data sources can connect, if your file share is visible, if that service answers. Maybe add application-calibrated load reporting. Beyond that, Jetty has good JMX support, but I’ve never needed it myself.
  • Load balancing: My setup supports no load balancing or failover out of the box. However, this is normally something that the web server or routers in front of the application server anyway. You might want to look into Jetty’s options for session affinity, if you need that.
  • Security: Jetty supports JAAS, of course. Also: In all the environments I’ve been working with (CA SiteMinder, Sun OpenSSO, Oracle SSO), the SSO server sends the user name of the currently logged in user as an HTTP header. You can get far by just using that.
  • Consistency: If you deploy more than one application as an embedded application server, the file structure used by an application (if any) should be standardized. As should the commands to start and stop the application. And the location of logs. Beyond that, reuse what you like, recreate what you don’t.

Taking control of your destiny

Using an embedded application server means using the application server as a library instead of a framework. It means taking control of your “main” method. There’s a surprisingly small number of things you need to work out yourself. In exchange, you get the control to do many things that are impossible with a big-A Application Server.

Reference: http://www.javaworld.com/community/

Tagged : / / / / / / / / / / / / / / /

Why Worry About Versioning? – Versioning Complete guide

versioning

Why Worry About Versioning?

Having a good version scheme for your software is important for several reasons. The following are the top five things a version scheme allows you to do (in random order):

Track your product binaries to the original source files.

Re-create a past build by having meaningful labels in your source tree.

Avoid “DLL hell” —multiple versions of the same file (library in this case) on a machine.

Help your setup program handle upgrades and service packs.

Provide your product support and Q/A teams with an easy way to identify the bits they are working with.

So, how do you keep track of files in a product and link them back to the owner? How can you tell that you are testing or using the latest version of a released file? What about the rest of the reasons in the list? This chapter describes my recommendations for the most effective and easiest way to set up and apply versioning to your software. Many different schemes are available, and you should feel free to create your own versioning method.

Ultimately, like most of the other topics in this book, the responsibility to apply versioning to the files in a build tends to fall into the hands of the build team. That might be because it is usually the build team that has to deal with the headaches that come from having a poor versioning scheme. Therefore, it is in their best interest to publish, promote, and enforce a good versioning scheme. If the build team does not own this, then someone who does not understand the full implications of versioning will make the rules. Needless to say, this would not be desirable for anybody involved with the product.

File Versioning

Every file in a product should have a version number; a four-part number separated by periods such as the one that follows seems to be the established best practice. There are many variations of what each part represents. I will explain what I think is the best way of defining these parts.

Major version— The component owner usually assigns this number. It should be the internal version of the product. It rarely changes during the development cycle of a product release.

Minor version— The component owner usually assigns this number. It is normally used when an incremental release of the product is planned instead of a full feature upgrade. It rarely changes during the development cycle of a product release.

Build number— The build team usually assigns this number based on the build that the file was generated with. It changes with every build of the code.

Revision— The build team usually assigns this number. It can have several meanings: bug number, build number of an older file being replaced, or service pack number. It rarely changes. This number is used mostly when servicing the file for an external release.

Build Number

Each build that is released out of the Central Build Lab should have a unique version stamp (also known as the build number). This number should be incremented just before a build is started. Don’t use a date for a build number or mix dates into a version number simply because there are so many date formats out there that it can be difficult to standardize on one. Also, if you have more than one build on a given date, the naming can get tricky. Stick with an n=n+1 build number. For example, if you release build 100 in the morning and rebuild and release your code in the afternoon, the afternoon build should be build 101. If you were to use a date for the build number, say 010105, what would your afternoon build number be? 010105.1? This can get rather confusing.

It is also a good idea to touch all the files just prior to releasing the build so that you get a current date/time stamp on the files. By “touching” the files, I simply mean using a tool (touch.exe) to modify the date/time stamp on a file. There are several free tools available for you to download, or you can write one yourself. Touching the files helps in the tracking process of the files and keeps all of the dates and times consistent in a build release. It also eliminates the need to include the current date in a build number. You already have the date by just looking at the file properties.

In addition, try to avoid tools that inject version numbers into binaries (as a post-build step). Although the tools might seem reliable, they introduce an instability factor into your released binaries by hacking hexadecimal code. Most Q/A teams become distressed if this happens to the binaries they are testing—and justifiably so. The build number should be built into the binary or document files.

Source Code Control Trees

All the source code control (SCC) software that I have seen has some kind of labeling function to track either checked-in binaries or source code. (Remember that I am not for checking in binaries, but I mention this for the groups or teams that do. This type of versioning (or more appropriately named labeling) is typically used to track a group of sources that correspond to a product release. Most of the time, they combine labeling of the sources with the branching of the source code lines.

Should There Be Other Fields in the File Version Number?

I am of the opinion that no, there shouldn’t be other fields in the file version number. Let’s look at some other fields that might seem like they should be included but really don’t need to be:

  • Virtual Build Lab (VBL) or offsite development group number— You can use this number to track a check-in back to a specific site or lab that the code was worked on. If you have a golden tree or mainline setup, and all your VBLs or offsite trees have to reverse integrate into the golden tree, this extra field would be overkill. That’s because you can trace the owner through the golden tree check-in. Having a field in which you would have to look up the VBL or offsite number would take just as long.

    The reality is that when you check a version number, you won’t care where the file came from. You’ll only care if it is a unique enough number to accurately trace the file to the source that created it. Most likely, you’ll already know the version number you’re looking for and you’ll just need to confirm that you’re using it.

  • Component—If you break your whole project into separate components, should each component have its own identification number that would be included in the version string? No, similarly to the reasons in the previous bullet, you can track this information by the name of the binary or other information when checking the other properties of the file. This information would probably only come into play if you were filing a bug and you had other resources available to determine which component this file belonged to.
  • Service Pack Build Number— If you’re doing daily builds of a service pack release, you should use the earlier example; increment the build number and keep the revision number at the current in-place file build number. This seems like a good argument for a fifth field, but it isn’t if the revision field is used properly.

 

DLL or Executable Versions for .NET (Assembly Versions)

This section applies to you only if you are programming in .NET.

When Microsoft introduced .NET, one of its goals was to get rid of DLL hell and all the extra setup steps I talk about later in this chapter. In reviewing where .NET is today, it looks like Microsoft has resolved the sideby-side DLL hell problem, but now the problem is “assembly version hell.” Without going into too much detail about the .NET infrastructure, let’s touch on the difference between file versioning and assembly versioning.

Assembly versions are meant for binding purposes only; they are not meant for keeping track of different daily versions. Use the file version for that instead. It is recommended that you keep the assembly version the same from build to build and change it only after each external release. For more details on how .NET works with these versions, refer to Jeffrey Richter’s .NET book. Don’t link the two versions, but make sure each assembly has both an assembly version and a file version when you right-click. You can use the same format described earlier for file versioning for the assembly version.

How Versioning Affects Setup

Have you ever met someone who reformats his machine every six months because “it just crashes less if I do” or it “performs better?” It might sound draconian, but the current state of component versioning and setup makes starting from scratch a likely solution to these performance issues, which are further complicated by spyware.

Most of the problems occur when various pieces of software end up installing components (DLLs and COM components) that are not quite compatible with each other or with the full set of installed products. Just one incorrect or incorrectly installed DLL can make a program flaky or prevent it from starting up. In fact, DLL and component installation is so important that it is a major part of the Windows logo requirement.

If you are involved in your product’s setup, or if you are involved in making decisions about how to update your components (produce new versions), you can do some specific things to minimize DLL hell and get the correct version of your file on the machine.

Installing components correctly is a little tricky, but with these tips, you can install your components in a way that minimizes the chance of breaking other products on your own.

Install the Correct Version of a Component for the Operating System and Locale

If you have operating system (OS)-specific components, make sure your setup program(s) check which OS you are using and install only the correct components. Also, you cannot give two components the same name and install them in the same directory. If you do, you overwrite the component on the second install on a dual-boot system. Note that the logo requirements recommend that you avoid installing different OS files if possible. Related to this problem is the problem caused when you install the wrong component or typelib for the locale in use, such as installing a U.S. English component on a German machine. This causes messages, labels, menus, and automation method names to be displayed in the wrong language.

Write Components to the Right Places

Avoid copying components to a system directory. An exception to this is if you are updating a system component. For that, you must use an update program provided by the group within Microsoft that maintains the component.

In general, you should copy components to the same directory that you copy the EXE. If you share components between applications, establish a shared components directory. However, it is not recommended that you share components between applications. The risks outweigh the benefits of reduced disk space consumption.

Do Not Install Older Components Over Newer Ones

Sometimes, the setup writer might not properly check the version of an installed component when deciding whether to overwrite the component or skip it. The result can be that an older version of the component is written over a newer version. Your product runs fine, but anything that depends on new features of the newer component fails. Furthermore, your product gets a reputation for breaking other products. We address the issue of whether it makes sense to overwrite components at all. But if you do overwrite them, you don’t want to overwrite a newer version.

“Copy on Reboot” If Component Is in Use

Another common mistake is to avoid dealing with the fact that you cannot overwrite a component that is in use. Instead, you have to set up the file to copy on reboot. Note that if one component is in use, you probably should set up all the components to copy on reboot. If you don’t, and if the user doesn’t reboot promptly, your new components could be mixed with the old ones.

Register Components Correctly; Take Security into Account

Sometimes setups don’t properly register COM components correctly, including the proxy and stub. Note that Windows CE requires that you also register DLLs. Note, too, that when installing DCOM components, you must be vigilant about permissions and security.

Copy Any Component That You Overwrite

It is smart to make a copy of any component that you overwrite before you overwrite it. You won’t want to put it back when you uninstall unless you’re sure that no product installed after yours will need the newer component—a difficult prediction! But by storing the component in a safe place, you make it possible for the user to fix his system if it turns out that the component you installed breaks it. You can let users know about this in the troubleshooting section of your documentation, the README file, or on your Web site. Doing this might not save a call to support, but it does at least make the problem solvable. If the component is not in use, you can move it rather than copying it. Moving is a much faster operation.

Redistribute a Self-Extracting EXE Rather Than Raw Components

If your component is redistributed by others (for instance, your component is distributed with several different products, especially third-party products), it is wise to provide a self-extracting EXE that sets up your component correctly. Make this EXE the only way that you distribute your component. (Such an EXE is also an ideal distribution package for the Web.) If you just distribute raw components, you have to rely on those who redistribute your components to get the setup just right. As we have seen, this is pretty easy to mess up.

Your EXE should support command-line switches for running silently (without a UI) and to force overwriting, even of newer components, so that product support can step users through overwriting if a problem arises.

If you need to update core components that are provided by other groups, use only the EXE that is provided by that group.

Test Setup on Real-World Systems

If you’re not careful, you can install all your setup testing on systems that already happen to have the right components installed and the right registry entries made. Be sure to test on raw systems, on all operating systems, and with popular configurations and third-party software already installed. Also, test other products to make sure they still work after you install your components.

Even Installing Correctly Does Not Always Work

Even if you follow all the preceding steps and install everything 100 percent correctly, you can still have problems caused by updating components. Why? Well, even though the new component is installed correctly, its behavior might be different enough from the old component that it breaks existing programs. Here is an example. The specification for a function says that a particular parameter must not be NULL, but the old version of the component ran fine if you passed NULL. If you enforce the spec in a new version (you might need to do this to make the component more robust), any code that passes NULL fails. Because not all programmers read the API documentation each time they write a call, this is a likely scenario.

It is also possible for a new version to introduce a bug that you simply didn’t catch in regression testing. It is even possible for clients to break as a result of purely internal improvements if they were relying on the old behavior. Typically, we assume that nothing will break when we update a component. In fact, according to Craig Wittenberg, one of the developers of COM who now works in the ComApps group at Microsoft, if you don’t have a plan for versioning your components in future releases, it is a major red flag for any component development project. In other words, before you ship Version 1, you need to have a plan for how you will update Version 1.1, Version 2, and beyond—besides how to bug-fix your updates.

Reference: The Build Master: Microsoft’s Software Configuration Management Best Practices

 

Tagged : / / / / / / / / / / / / / / / / /