Continuous Integration or CI is a practice used by the developers which requires every single developer in the team to integrate his/her code into a shared repository several times a day. Each individual check-in is always verified by an automated build process, thus allowing the team to detect the problems at an early stage. This automated build script can also include executing the test suite for the entire application, thus ensuring that the code which is in the repository is a stable version. In this document we will talk about the Continuous Integration Process and Jenkins as a CI tool in detail. This document is prepared to address the group of readers or developers who believe in the Automation mechanism.
Continuous Integration or CI is also defined as a software engineering practice in which isolated small business logic changes are immediately tested and reported on when they are included into a larger code base.
Continuous Integration Tools available
Before going into further details, let us have a look at the list of tools that are recognised and used as Continuous Integration tool -
- Cruise Control - Cruise Control is a free continuous integration tool which is aimed to serve the Java or JEE technologies. On top of this, there is a version designed for .NET technology as well named 'Cruise Control .Net'. In Cruise Control, the entire configuration is done with the help of an XML file. In addition to the XML file, we have a web based dashboard component which is used to view the build progress. As compared to the other tools, Cruise Control does not have a configuration interface. Thus we should use some special editors which can work with the Cruise Control's config file if we don't like editing the XML file by hand.
- Continuum - Continuum is an open source continuous integration tool from Apache which is primarily designed and developed for Java builds. This tool comes up with a unique feature - that is the role-based security. This feature gives us the control on certain aspects of the build which can be seen by specific users. All other features are similar to Hudson and Cruise Control e.g. the release management feature, email notification feature, integration with popular build tools e.g. Ant, Maven etc. It can also integrate with source control management systems e.g. SVN or CVS. This also has a web based dashboard which can be accessed via URL.
- Hudson - Hudson is a free continuous integration tool which is designed for Java and related JEE technologies. It's relatively new as compared to its competitors, but it has the best user interface. Similar to Continuum, Hudson also has a very short learning curve. Installation of Hudson is even simpler and easier than Continuum, and requires the execution of Java common once the library (jar) file is placed on the machine which is the server.
- Jenkins - Jenkins is an open source continuous integration tool which is also written in Java. The project got branched out from Hudson as there was a dispute with Oracle. Jenkins provides continuous integration services for JAVA/JEE related applications. It is a JEE based application which can easily run on any servlet container e.g. Apache Tomcat. This supports the commonly used SCM tools which include AccuRev, CVS, Subversion, Git, Mercurial, Perforce, Clearcase and Relational Team Content or RTC. It has the ability to execute Apache Ant and Apache Maven based projects. At the same time it can also execute the arbitrary shell scripts and Windows batch commands.
- Bamboo - Bamboo is a continuous integration server from Atlassian. Atlassian is the same company who has provided JIRA, Confluence and Crowd. Bamboo can be used without any cost for philanthropic and open-source projects, while the commercial organizations are charged based on the number of required build agents. Bamboo supports builds in any programming language using any build tool, including Ant, Maven, make, and any other command line utilities. Build notifications can be customized based on the event type, and received via email, instant message, RSS feeds, or pop-up windows in Eclipse-based IDEs or IntelliJ IDEA.
Mechanisms in Continuous Integration
Continuous Integration (CI) is followed as a development practice wherein the developers are required to integrate their code into a shared repository multiple times a day. Each and every check-in is always verified by build process (automatic). This allows the teams to detect problems at an early stage.
Since you are integrating very frequently, there is a less possibility of things going drastically wrong. Thus the developers and the team can spend more time in developing the features smartly rather than solving build issues.
Continuous Integration process is very simple.. If it is not implemented, the cost goes higher. If we do not follow a continuous integration approach, we need to spend periods between integrations. This also makes it exponentially more difficult to find and fix the problems. This issue becomes critical at times and can easily derail a project from the schedule, or even cause it to fail altogether.
Continuous Integration comes up with multiple benefits to our organization. These are listed below -
- No need to indulge in long and tense integration processes.
- The visibility increases which enables greater communication.
- Catch the issues early and kill them in the bud.
- Spend less time in debugging and more time in development.
- It gives a confidence to the senior management about the product.
- We do not need to wait to see if our code is going to work or not.
- It reduces integration challenges, thus allows us to deliver software more rapidly.
Continuous integration is more than a process. It is supported by several important processes and principles and practices. These practices are listed below -
- Maintain a single source repository
- Automate the build
- Make your build self-testing
- Every commit should build on an integration machine
- Keep the build fast
- Test in a clone of the production environment
- Make it easy for anyone to get the latest executable
- Everyone can see what is going on
- Automated deployment
These practices can be achieved by taking the following steps -
- Developers check out code into their private workspaces.
- When done, the commit changes to the repository.
- The Continuous Integration server monitors the repository and checks out changes when they occur.
- The Continuous Integration server builds the system and runs unit and integration tests.
- The Continuous Integration server releases deployable artefacts for testing.
- The Continuous Integration server assigns a build label to the version of the code it just built.
- The Continuous Integration server informs the team of the successful build.
- If the build or tests fail, the Continuous Integration server alerts the team.
- The team fix the issue at the earliest opportunity.
- Continue to continually integrate and test throughout the project.
The responsibilities of the development team members are listed below -
- Check in at a regular interval.
- Do not check in broken or un-compiled code
- Do not check in untested code
- Do not check in when the build is broken
- Do not go home after checking in until the system builds
Continuous Deployment is very closely associated to Continuous Integration. It refers to the release of the software to production which has passed the automated tests. The practice is to release a good quality code to the end users. As explained by Jez Humble, author of Continuous Delivery.
Using Continuous Integration and Continuous Deployment not only reduces the risks and catches bugs quickly, but also releases the working software very rapidly.
With risk free releases, you can quickly adapt the new business requirements and the customer needs. This allows for higher collaboration between the ops and delivery. Thus, fuelling real change in the organisation, and converting your release process into a business advantage.
Key Principles of Continuous Integration
Following are the key principles of Continuous integration.
Revision Control is the basic standard which any software development team is following for the last couple of decades. The purpose of revision control which is also termed as version control or source control is to manage the changes made in the files that make up the code for a software project. It also allows us to switch back and forth within different versions of code using a single command, provided the changes are committed properly. In short, Revision Control not only makes our code base manageable while dealing with making/moving between changes, it also makes it much easier for the developers to integrate their code. This is the root of Continuous Integration; it is the best investment though most version control systems are free that an organization can make to ensure a cool life of the developers.
Build Automation is the ability to either automatically trigger and/or build a clean version of the entire source code using a single command or action using raw components. Most teams now take this further and allow their systems to be built completely from the code which is obtained directly from their version control system. This process has the ability to take a 30 minute task and wrap it up in few seconds. Establishing the patterns that the developers perform to start a clean or new system and also the environment for which the build is to be built, e.g. whether this be a production, staging, or development build saves our time in the long term.
Automated Deployment is the process in which we consistently keep on pushing a product to various environments with the help of some “trigger”. This allows us to quickly learn what is to be expected every time we deploy an environment having a much faster results. This combined with Build Automation has the ability to save a significant amount of hours for the development teams.
Automated Deployment saves the customers from being extensively offline while deployment and also it allows the developers to build with disturbing less number of clients systems. With an automated system, human error is prevented. In the event of human error, developers are able to catch it before live deployment
Testing can happen in multiple ways using the methods which we have probably heard as buzzwords. Some of these include automated testing e.g. unit testing and interface testing. Self-testing Builds are the next step. Once you have tests which can be automated, executing them simultaneously with a build is usually an easy job. The teams should have more awareness on what is going on with the product. Obviously, the more we test the greater visibility we have on the status of our product before the release. This is probably the difference between being a reactive or a proactive. It must be remembered that: Testing is only as good as the environment in which you test it.
Testing in a Clone of Production
It is common practice that every developer has the story of burning the proverbial midnight oil since their build didn't work on the production environment while the same is perfectly running on the staging or development environments. We must ensure that our testing environment is as close to production as possible. This approach minimizes this risk.
Whenever we commit a code we are creating a new version in the version control system. This marks a point in the history of the code base where anyone is allowed to switch over to that point and use the changes made. In fact, each and every release of the product may include thousands of commits. This is a very good approach - the more modular the commits, the easier it is for developers to merge, move, add, and remove changes as per their need. This approach also gives a point of comfort, because if something does go wrong, they can start back where they were the last time they committed work.
Code Consolidation is typically the best thing at the end of the day. This is the time when all development features are compiled and merged. It has some good features: -
- Developers need to spend less time trying to merge their changes with one another.
- If the Automated Build is enabled on a test server and those builds are Self-testing Builds, we should be able to track the performance of your product throughout production.
- It gives visibility to all the developers across the team. Every developer is now aware of what code other developer is writing.
Time is money. Faster Builds saves lot of time. Faster Build mechanism allow the developers to get on their feet faster, receive test results faster and see the product faster.
It is very important to have a current development and staging build available at all times. Regardless of the Automated Testing, there are few things which a computer cannot detect
Test Result Availability
It is very important to check which test passed and which failed. Availability of the Test Result gives us the visibility on the results to developers, managers and client stakeholders.
Setting up Jenkins
As a standard practice followed in the open source technology, we will setup Jenkins on Community ENTerprise Operating System or CENT OS. In order to set up we need to go thru the following steps -
Step 1 - We first need to ensure that JDK is installed on the system. Verify this by using either of the following commands -
Listing 1 - Check the Java version
If JRE is installed on your system, you should get the following output -
java version "1.7.0_72" Java(TM) SE Runtime Environment (build 1.7.0_72-b14) Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)
Or you can use the following -
Listing 2 - Check the Java version
If JDK is installed on your system, you should get the following output -
Either of the above two commands ensure that Java is installed properly on your system. If Java is not installed then run the following command to get it installed -
Listing 3 - Install Java
yum install java-1.7.0_72-openjdk
This will install the open JDK on your system. If you want to install Sun (Oracle) Java, down the rpm file from the official Oracle site as per your environment and install it using the standard rpm command.
Step 2 - Once done with the Java part, next step is to add the Jenkins repository to your system using the following -
Listing 4 - Add Jenkins repository to your computer
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
Then run the following command to install Jenkins -
Listing 5 - Add Jenkins repository to your computer
// code for computer yum install jenkins
Step 3 - After Jenkins is installed, add this in the system boot using the following command -
Listing 6 - Add Jenkins to system boot
// code for system boot yum install jenkins
Step 4 - Now start Jenkins using the following
Listing 7 - Add Jenkins to system boot
service jenkins start
By default, Jenkins runs on port 8080. So once started, go to your browser and type the following URL -
Listing 8 - Jenkins URL
You should get the following page on your browser -
Figure 1. Jenkins Dashboard
User Authentication in Jenkins
By default Jenkins comes with all user Authentication mechanism which means any one can do anything. To have controlled access, we need to have a role based authentication mechanism. We need to follow these steps to setup role based authentication mechanism -
- Step 1 - To start with, we need to go to the settings sections in the dashboard and then navigate to the following link -
Click on Manage Jenkins >> Configure Global Security >> Enable Security >> Jenkins Own User Database >> Allow Users to Signup >> Save.
- Step 2 - Once the above setting is saved, we can start adding users. To add a user click on the 'Sign Up' button on the top right corner of the dashboard page.
- Step 3 - Now let us restrict User Signup and set the “Matrix based security authentication”. To achieve this we need to go thru the below steps :-
- Navigate to the link - Manage Jenkins >> Configure Global Security >> Enable Security >> Jenkins Own User Database >> Uncheck “Allow Users to Signup” >> Matrix-based security >> Use the Put the user Name @ “User/group to add:” text box >> click on add >> Check all the Check boxes >> Save
- Use the user name as 'Chris West'. So, now onwards, the user 'Chris West' will be administrator of this Jenkins setup.
- Click on Manage Jenkins > Manage Users > Create Users. Follow the steps in step 2 & 3 to add users.
Once the uses are added, you can see the following screen -
Figure 2. Jenkins Dashboard - List of users
Configure JDK and other build utilities
Once the user authentication part is over, we need to set up Java and other build utilities. To do this we need to navigate to the link - Manage Jenkins >> Configure System. Here you will get the UI to enter the path for JDK, ANT and Maven. Set the appropriate values as per your environment.
Adding Credentials and nodes
We need to add a credential before adding a node. To do this login into Jenkins using your credentials and click on the credentials tab. Then click on the 'Add Domain'. Here, provide a name to this domain and some description which describes the name properly. Finally, hit the 'OK' button. Now click on the 'Add Credentials' and provide the necessary inputs here. Once the credentials are added, we will now see how to add a node. Go to the main dashboard of Jenkins Click on Manage Jenkins >> Click on Manage Nodes from the List. You should find a Node named Master in the node list which is the Jenkins server itself. To add a new node, click on 'New Node'. Enter the information and click on OK. On the next screen, enter all the information about your host. Click on the advance and change the port number if you are using any port which is different from the standard for your SSH. Once all the parameters are set hit the Save button. It might take some time to launch the new node. Once launched, you should be able to see all the information related to you node that you have entered earlier.
Plugin Management with Jenkins
Jenkins has the ability to add plugins to have more functionality. We can configure these plugins via the link - Manage Jenkins >> Manage Plugins. Commonly used plugins are –
Allows to connect to GIT SCM
Allows to connect to SVN SCM
Allows projects to run xvnc during a build. This allows for example to run tests which requires a display to run on a virtual display.
Setup SVN plugin
In order to setup the SVN plugin on jenkins, navigate through - Manage Jenkins >> Manage Plugins and then select the SVN plugin here. Click on the 'Download and install after restart' button. This will download the svn plugin. Once the download is over restart the jenkins using
Listing 9 - Restart Jenkins
service jenkins stop service jenkins start
On successful restart, Jenkins is ready to be used with SVN. You can point your SVN repo from Jenkins and trigger the build from here itself.
Through this document, I have tried to cover the Continuous integration process and the different tools that are available. I have also discussed Jenkins which is a relatively new Continuous Integration tool. Let us summarize our discussion in following bullets -
- Continuous Integration or CI is a practice used by the developers which requires every single developer in the team to integrate his/her code into a shared repository several times a day.
- Following tools are available for Continuous Integration or CI -
- Cruise Control
- Continuous Integration is based on the following principles -
- Revision Control
- Build Automation
- Automated Deployment
- Self-Testing Builds
- Testing in a Clone of Production
- Frequent Commits
- Code Consolidation
- Fast Builds
- Build Availability
- Test Result Availability
- Jenkins is an open source continuous integration tool which is also written in Java.
- The project got branched out from Hudson as there was a dispute with Oracle.
- Jenkins provides continuous integration services for JAVA/JEE related applications.
We hope you have enjoyed reading this article.