Regression Testing in Agile Scenario
Summarize With:
An increasing number of software development teams today favor agile software development over other development approaches. There's no need to introduce Agile, but let's revisit it anyway! We know that the Agile development methodology is adaptive in nature. It focuses on building a product based on empirical feedback gathered from frequent testing by breaking down the product development process into shorter development cycles or sprints. Key is that since agile development is iterative, all the aspects of product design are frequently revisited to ensure that change can be effected as soon as the need for it is identified. This is where we segue into testing — clearly in this scenario, developers need to test more and test fast. To do that, curating a comprehensive test plan based on the agile methodology is essential.
In agile development, testing needs to grow geometrically with each sprint and testers need to ensure that the new changes implemented do not affect the other parts of the application and existing functionalities — we know this type of testing as Regression Testing and this holds a very important place in the agile development scenario.
Regression testing essentially checks if the previous functionality of the application is working coherently and that the new changes executed have not introduced new bugs into the application. These tests can be implemented on a new build for a single bug fix or even when there is a significant change executed in the original functionality. Since there can be many dependencies in the newly added and existing functionalities, it becomes essential to check that the new code conforms with the older code and that the unmodified code is not affected in any way. In agile development, since there are many build cycles, regression testing becomes more relevant as there are continuous changes that are added to the application.
Few Points to be considered before Regression Test Plan
For effective regression testing in agile development, it is important that a testing team builds a regression suite right from the initial stages of software development and then keeps building on it as sprints add up. A few things to determine before a regression test plan is built are:
- Identifying which test cases should be executed.
- Identifying what improvements must be implemented in the test-cases.
- Identify the time to execute regression testing.
- Outline what needs to be automated in the regression test plan and how.
- Analyze the outcome of the regression testing
Along with this, the regression test plan should also take into account Performance Testing to ensure that the system performance is not negatively affected due to changes implemented in the code components.
In the agile environment, Regression Testing is performed under two broad categories;
- Sprint level Regression testing: This regression test is focused on testing the new functionalities that are implemented since the last release.
- End to End Regression testing: This test incorporates end-to-end testing of 'all' the core functionalities of the product.
Considering that the build frequency in agile development is accelerated, it is critical for the testing team to execute the regression testing suite within a short time span. Automating the regression test suite makes sense here to ensure that the testing is completed within a given time period and that the tests are error free. Adding the regression testing suite to the continuous integration flow also helps as this prevents the developers to check-in new code before automatically evaluating the correct working of the existing functionality.
Regression testing can adopt the following approaches:
- The Traditional Testing Approach: In this method, each sprint cycle is followed by sprint level regression test. Post a few successful sprint cycles, the application goes through one round of end-to-end regression testing.
This method allows the team to focus on the functional validity of the application and gives testers the flexibility to decide on the degree of automation they want to implement. - Delayed Week Approach: In this approach, the sprint level regression test is not confined to a timeline and can spill over into the next week. For example, if the sprint level regression test that is supposed to be completed in Week 2 is not completed, then it can spill over to Week 3. This approach works well in the beginning of the testing cycles as the testing team at that time is still in the process of gaining an implicit understanding of the functionalities and the possible defects. Instead of parking the bugs/defects and addressing them at a later date, continuous testing lifts the burden of backlogs that build up during end-to-end regression tests.
- Delayed Sprint Approach: In this approach, the regression cycle is common and regression test cases that were employed for the second sprint contain the functionality stories that were a part of the first sprint. Since the regression cycle is only delayed by a sprint, this approach discounts the need for having two separate regression test cycle types. This approach also avoids a longer end-to-end regression test cycle. However, this approach has two challenges —
- Maintaining the sanctity of the regression tests is difficult.
- Maintenance of automation efforts increases considerably.
Conclusion
Organizations need to decide on their approach to regression testing keeping the continuity of business functions in mind. Regression tests target the end-to-end set of business functions that a system contains and are conducted repeatedly only to ensure that the expected behavior of the system remains stable. Our view is that having a zero tolerance policy towards bugs found in regression tests makes agile development more effective and ensures that the changes implemented in the product do not disrupt the business functions. We are not among those who agree with Linus Torvalds in his assessment, 'Regression Testing? What's that? If it compiles it is good; if it boots up, it is perfect.'