"If you automate a mess, you get an automated mess.' (Rod Michael)
Virtually all software development projects today contain an element of test automation. If an initiative is not already underway chances are there is one under consideration. The software engineering groups are generally convinced of the benefits — greater test coverage achieved faster being the most apparent. That being said as the business owner how does the economic argument in favour of test automation stack up?
Before that- is there any value, though, in getting into ROI discussions? The point being that in all software development, testing is a given. Despite that has there ever been any meaningful attempt to try & define the ROI of the testing effort? The fact is that the risk of sending an untested product into the market is just too great for the business to bear so an ROI argument is generally not necessary. So let us start with the base assumption that a certain cost for testing is inevitable — this would likely be the cost allocation made towards the efforts the testers put in to achieve the desired quality standards before the product or software release.
Let us consider the various cost elements of a test automation initiative.
The cost of the automation effort is a combination of all of these and the cost to be considered in any, even notional, ROI discussions would be that, less the cost allocation that would have been made towards a traditional manual testing effort. Are you with us so far?
Having dealt with the, I in ROI let's move towards the R. We spoke of the principal benefits from automating your software testing — more coverage and faster testing. What are the possible economic benefits that can accrue from these, though?
First, let's look at the greater coverage. The conventional wisdom is that greater code coverage will uncover more defects and potentially earlier in the development cycle.
What is the value of testing faster? It seems reasonable to assume that faster testing will shorten development cycle time and, therefore, allow for faster releases, more product iterations within a shorter time and so on. How does that look on the balance sheet?
One way to look at this argument is that it is too complex to look at purely from a 'cash in, cash out' kind of viewpoint so you should consider only the engineering value. The variables are many but isn't that in many ways true of software testing in general. Remember the words of James Bach,
'Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the unthinkable happening to the anonymous.'
Share This Article: