Whilst searching for ways to measure the value of test automation, I read Doug Hoffman’s (I’m sure classic) Cost Benefits Analysis of Test Automation paper.
The first several pages were great. He discussed intangibles that should be left out of an ROI calculation like an immediate reduction in perceived productivity of the test organization as the automation is first developed. He went on to list the falsely expended benefits like the automation of existing manual tests. Then he compared fixed automation costs like scripting tools to variable automation costs like test maintenance.
Finally, Doug got to the formulas. After careful analysis of some 30+ factors, one can start calculating automation ROI and efficiency benefits. I rubbed my hands together and excitedly turned the page. Then…I think I puked into my mouth a little as I saw the following:
In the end, I latched onto one powerful statement Doug made, almost in passing, he said:
If the benefits of automation are required, then the ROI computation is unnecessary, the investment is required…it’s an expense.
If the benefits include reducing risk by performing automated checking that would not be possible by humans (e.g., complex math, millions of comparisons, diffs, load, performance, precision), then say no more…
I don’t want to suffer through the computations.