Fundamentals of Continuous Integration Testing

Over the past decade, most organizations have adopted agile development methodologies to develop products faster and reduce their time to market. As agile development methodologies have evolved they have given rise to newer development practices that put testing at the heart of development. Continuous Integration is one such development practice that adopts an architecture based approach, to provide development teams more flexibility and ensure the production of high-quality software even more frequently.

Testing and development expert, Martin Fowler defines Continuous Integration as a practice where a development team integrates their work on a daily basis and makes way for multiple daily integrations, which are then verified by an automated build and test. This allows for faster error detection and significantly reduced integration errors. This makes the development process more efficient. Adopting the Continuous Integration methodology enables teams to develop software that is more cohesive and robust, even while releasing at extremely short intervals.

One of the main contributors to Continuous Integration success is continuous testing. Delivering high-quality software in extremely short timeframes is not possible unless each and every piece of code is tested thoroughly. Testing, thus, becomes one of the most important elements of Continuous Integration. The development team also has to work on developing a comprehensive automated testing suite both at the unit and at the functional level to guarantee code quality.

Since the goal of Continuous Integration is to never break the build, it becomes imperative that all measures are taken to ensure that untested or broken code does not get committed and that strict version control policies are implemented. The entire environment here has to be based on automation to ensure that despite any and every code addition to the application results in a releasable version. Additionally, it also has to ensure that the application can be built in any version.

What is Continuous Integration?

Continuous Integration is a subset of Continuous Delivery, where the built application is delivered to the testing and production teams to ensure application reliability, generate faster user feedback and ensure continued high performance. Continuous Integration automates the manual stages of application development and hence, makes the development and deployment process faster. Frequent and incremental builds are the hallmark of Continuous Integration and this effectively eliminates the requirement of integration testing.

In Continuous Integration, the developers submit the new code and/or code changes to a central code repository where the release manager merges the code with the main branch and then pushes out the new release. It is the Continuous Integration system that monitors the version control system for changes and then launches the build after getting the source code from the repository. The server then runs unit tests and functional tests to check the functionality, validity, and quality of the product. The same package is then deployed for acceptance testing and is then deployed on the production server.

Automation plays a critical role in Continuous Integration to ensure that all packages can be deployed on all the servers at the click of a button. To enable this, its becomes essential to maintain a single source code, automate the build, keep the build fast and make it self-testing. Along with this, every commit has to be built on the integration machine, keeping systems transparent and easily accessible by the invested parties to test the product in a production environment. The Continuous Integration server has to competently inform the respective teams of each successful build and alert the team in case of any failure. The team then has to ensure that the issue is fixed at the earliest.

It is clear that with Continuous Integration, testing, too, is continuous. Some key testing areas to focus on are:

• Ensure that continuous regression tests are run in the background and that they capably provide regular feedback to minimize regression defects.
• Carry out continuous performance tests to study the application for response time, identify changes in speed, reaction time, and application consistency.
• Conduct frequent load tests to clear performance goals and make the application ready for use.
• Ensure that the load tests begin in smaller and incremental scenarios and culminate as one large package.
• Conduct continuous scalability testing to gauge the throughput, network, and CPU memory usage, to reduce business risks.
• Ensure end-to-end functional tests to test the functionality of the product in different scenarios.

Continuous Integration sets the stage for Continuous Delivery by reducing manual intervention. It also brings greater transparency to the entire development and QA process which makes taking factual decisions to improve efficiency faster. Since Continuous Integration testing aims to find bugs and code breaks faster with the help of more automation, the development process becomes more cost efficient. Steve Brodie, CEO of Electric Cloud aptly summarizes the importance of automation in a Continuous Integration and Continuous Delivery world as, 'You're only as fast as the slowest element, the slowest phase in your software delivery pipeline'. Clearly, automation lies in the heart of the Continuous Integration process and makes the magnitude of change implementation smaller, more manageable and efficient.