The first time I saw James Whittaker was in 2004 at an IIST conference. He dazzled us with live demos of bugs that were found in public software. This included a technique of editing HTML files to change quantity combo box values to negatives, resulting in a credit to one’s VISA card.
And now, fresh into his job as Google’s test engineering director, I was thrilled to see him headlining STARwest’s keynotes with his challenging “All That Testing Is Getting in the Way of Quality” presentation.
After a brief audience survey, James convinced most of us that testers do not get respect in their current roles. Then he kicked us while we were down, by suggesting the reason we lack respect is all the recent software quality “game changers” have been thought of by programmers:
- Today’s software is easier to update and fix problems in (easier than software from 10 years ago).
- Crash recovery – some software can fix itself.
- Reduction of dependencies via standards (e.g., most HTML 5.0 websites work on all browsers now).
- Continuous Builds – quicker availability of builds makes software easier to test but it has nothing to do with testers.
- Initial Code Quality (e.g., TDD, unit tests, peer reviews)
- Convergence of the User and Test Community (e.g., crowd source testing, dog food testing). Per James, “Testers have to act like users, users don’t have to act.”
Following the above were four addition “painful” facts about testing:
- Only the software matters. People care about what was built, not who tested it.
- The value of the testing is the activity of testing, not the artifact. Stop wasting your time creating bug reports and test cases. Start harnessing the testing that already exists (e.g., beta testing).
- The only important artifact is the code. Want your tests to matter? Make them part of the code.
- Bugs don’t count unless they get fixed. Don’t waste time logging bugs. Instead, keep the testers testing.
The common theme here is that programmers are getting better at testing, and testers are not getting better at programming. The reason this should scare testers is, per James:
“It’s only important that testing get done, not who gets it done.”
I agree. And yes, I’m a bit scared.
After a cocky demo of some built in bug reporting tools in a private version of Google Maps, James finally suggested his tip on tester survival; get a specialty and become an expert in some niche of testing (e.g., Security, Internationalization, Accessibility, Privacy) or learn how to code.
The hallway STARwest discussions usually brought up Whittaker’s keynote. However, apart from a few, nearly everyone I encountered did not agree with his message and some even laughed it off. One tester I had lunch with tests a system used by warehouse operators to organize warehouses. He joked that his warehouse users would not be drooling at the opportunity to crowdsource test the next version of WarehouseOrganizer 2.0. In fact, they don’t even want the new version. Another tester remarked that his software was designed to replace manual labor and would likely result in layoffs...dog food testing? Awkward.
Thank you STARwest, for bringing us such a challenging keynote. I thoroughly enjoyed it and think it was an important wake up call for all of us testers. And now I’m off to study my C#!