Don’t Test It #1 - Crisis In Production
9 comments Posted by Eric Jacobson at Friday, March 25, 2011I find it belittling…the notion that everything must be tested by a tester before it goes to production. It means we test because of a procedure rather than to provide information that is valuable to somebody.
This morning our customers submitted a large job to one of our software products for processing. The processed solution was too large for our product’s output. So the users called support saying they were dead in the water and on the verge of missing a critical deadline. We had one hour to deliver the fix to production.
The fix, itself, was the easy part. A parameter needed its value increased. The developer performed said fix then whipped up a quick programmatic test to ensure the new parameter value would support the users’ large job. Per our process, the next stop was supposed to be QA. Given the following information I attempted to bypass QA and release the change straight to production:
- Testers would not be able to generate a large enough job, resembling that in production, in the available time given.
- There was no QA environment mirroring production bits and data at this time. It would have been impossible to stand one up before the one hour deadline.
- The risk of us breaking production by increasing said parameter was insignificant because production was already non-usable (i.e., it would be nearly impossible for this patch to make production worse than it already was).
Even with the above considerations, some on the team reacted with horror…”What? No Testing?”. When I mentioned it had been tested by a developer and I was comfortable with said test, the response was still “A tester needs to test it”.
After convincing the process hawks it was not feasible for a tester to test, our next bottleneck was deployment. Some on the team insisted the bits go to a QA environment first, even though it would not be tested. This was to keep the bits in sync across environments. I agree with keeping the bits in sync, but how about worrying about that once we get our users safely through their crisis!
As I watched the email thread explode with process commentary and waited for the fix to jump through the hoops, I also listened to people who were in touch with the users. The users were escalating the severity of their crisis and reminding us of its urgency.
I believe those who insist everything must be tested by a tester do us a dis-service by making our job a thoughtless process instead of a sapient service.
- As with fishing, testing without catching bugs can get boring. It’s important to let newbies catch some bugs.
- When someone finds a bug you missed, it’s humiliating. When the tables turn and you find the missed bug, maybe it’s time to set your ego aside and put your coaching hat on.
Testing is Intangible - Ask Testers What They Did
4 comments Posted by Eric Jacobson at Wednesday, March 09, 2011- How did George Clooney do?
- Was it slow paced?
- Was the story easy to follow?
- Any twists or unexpected events?
- Was the cinematography to your liking?
Test Automation Authoring Is Harder Than It sounds
5 comments Posted by Eric Jacobson at Friday, March 04, 2011Problem Steps Recorder – Windows 7 Test Tool
3 comments Posted by Eric Jacobson at Tuesday, February 22, 2011
Programmer Profiling - Don't Be Afraid To Use It
5 comments Posted by Eric Jacobson at Friday, February 11, 2011- The TSA protects passengers by finding bombs among 3 million people.
- I protect users by finding bugs among 3 million lines of code.
- If I ask ProgrammerA how she tested something and she shows me a set of sound unit tests, and a little custom application she wrote to help her test better, I gain a certain level of confidence in her code.
- On the other hand, if I ask ProgrammerB how she tested something and she shrugs and says “that’s your job”, I gain a different level of confidence in her code.

RSS