In Software, What To Automate Is As Important As How To Automate
Summarize With:
If Shakespeare was a tester, in the initial days of test automation adoption he would certainly have asked, 'To automate or not to automate, that is the question'. As the adoption of test automation has increased and become an integral part of every testing strategy, this question has evolved just that little bit. Today, most testing teams recognize that they have to incorporate test automation to keep up with the speed of development.
Agile testing methodologies and newer software development methodologies such as Test Driven Development (TDD) etc. place testing at the heart of software development. Hence, the tests have to run as fast as the development process. A failure to do so will drive up costs due to timeline overruns.
While test automation carries the promise of great software quality, the fact remains that we cannot automate each and every test. Why?
Simply because you want to get maximum returns from your test automation initiatives. Automating everything only drives up costs because of the time and resources required and the level of complexity involved. At the same time, it is essential to note that by automating the right tests, teams can increase test coverage, reduce the number of bugs and improve software quality and eventually take your product to market much faster.
The reality is that automated testing is not an 'all or nothing' proposition. Software testing still needs some amount of manual testing — the trick to testing success lies in identifying what to automate as much as in deciding how to automate.
When to use Test Automation?
For any automation initiative to be successful it is imperative that the testing team first identifies the activities that are repetitive in the development cycle. Identifying the development environment and validating the functionality across these environments becomes the starting point of all automation initiatives. It\'s best to not compare automated and manual testing since both these activities serve a different purpose. With test automation, you can increase test coverage, get faster feedback, find more bugs and save time. Manual testing, on the other hand, essentially involves the checking of facts and thus becomes a more investigative exercise where tests are designed and executed simultaneously and the human brain is employed to spot failures in the system.
Automation tests take the pain out of testing by taking care of the tasks that are repeatable. In our experience, we have seen the below-mentioned tests that lend themselves beautifully to automation and increase test accuracy and improves software quality.
Regression Testing
Even the smallest tweak in software code can lead to the product behaving differently. When you fix something, you run the risk of breaking something else in the code. Regression testing ensures that any change of any addition to the software code does not impact the existing functionality. This test also catches bugs that might have been unwillingly released into the system due to an upgrade or a patch. During the course of software development, regression tests are conducted frequently to assess that even the smallest alterations, enhancement, configuration changes etc. in the application source code does not impact application functionality.
Functional Testing
Automating most, if not all of the functional tests also enhances the performance of a testing team. Functional Testing is focused on what the software 'does' and is not concerned with the internal details of the application. It, thus, becomes easier for the testers to automate tests and set performance benchmarks that are developer-independent to assess if the function being developed performs as expected and is crash resistant when faced with user load. Automating functional tests ensures that even the most inexperienced tester can perform powerful and comprehensive functional tests and contribute to developing a robust software product.
Unit Testing
Unit testing is the testing of small code fragments to gain a deeper and more granular view of how the code is performing. The identified pieces of code are checked independently and in isolation to ensure that they are behaving correctly. Manual unit testing is time and resource intensive and can be error prone. Automating unit tests ensures that the source code remains error free, identifies errors early in the development phase and ensures that the code developed works then and in the future.
Integration Testing
Integration testing is performed to see how the software performs when all the pieces are put together. Integrations tests are performed when there is a coupling between two software systems. When a coupling is broken the software does not perform as it should. Since integration testing involves testing across all layers, manually testing these would mean re-executing these tests by hand each time.
This impacts the build process negatively as manual testing of integration tests becomes extremely time-consuming and resource intensive. Instead, if integration tests are automated, testers will be able to catch bugs faster and ensure that the application is performing optimally when all its pieces are put together.
Smoke Testing
Smoke testing is a quick test conducted after a build is completed for identifying and then fixing obvious defects in a piece of software. Smoke tests are usually non-comprehensive and are focused on to assess that only the important functions work and assess if the build is stable enough to proceed with further testing. Smoke testing is also called Build Verification Testing and should be automated if builds are frequent as it exposes integration issues and identifies problems with the code early.
Performance Testing
Performance testing of an application is intensive and an exhaustive process as it involves identifying performance issues. Tests like load testing, volume testing, stress testing etc. fall within the purview of performance testing as these are all targeted towards identifying factors that affect the performance of an application.
Performance tests are conducted to assess that the application can manage varying degrees of system transactions, handle a large volume of concurrent users without impacting the speed, stability, and scalability of the application negatively. Performance testing involves the testing of several functional and non-functional components of the application and assesses the reliability of the product and identify reasons behind performance bottlenecks (whether it is software or hardware problem).
Conclusion
Having established the case for test automation, it is important to note that tests that check the application usability, random testing, device interface testing, back-end testing are generally best conducted manually. Employing great manual testers is essential especially during exploratory testing since the manual testers have the ability and experience to question the system and see if things behave differently. For complete testing success, it thus becomes essential to take a strategic approach towards the testing initiatives and find the balance between manual testing and test automation and then find the right set of testing tools to aid the automation process so that it becomes cost effective and delivers great returns.