I’m going back through my STARwest notes and I want to blog about a few other sessions I enjoyed.

One of those was Nancy Kelln and Lynn McKee’s, “Test Estimation and the Art of Negotiation”, in which they suggested a new way to answer the popular question…

How long will testing take?

I figured Nancy and Lynn would have some fresh and interesting things to say about test estimation since they hosted the Calgary Perspectives on Software Testing workshop on test estimation this year. I was right.

In this session, they tried to help us get beyond using what one guy in the audience referred to as a SWAG (Silly Wild-Ass Guess).

  1. Nancy and Lynn pointed to the challenge of dependencies. Testers sometimes attempt to deal with dependencies by padding their estimates. This won’t work for Black Swans, which are the unknown unknowns; those events cannot be planned for or referenced.
  2. The second challenge is optimism. We think we can do more than we can. Nancy and Lynn demonstrated an example of the impact of bugs on testing time. As more bugs are discovered, and their fixes need to be verified, more time is taken from new testing, time that is often under estimated.
  3. The third challenge is identifying what testing means to each person. Does it include planning? Reporting? Lynn suggested to try to estimate how much fun one would have at Disneyland. Does the trip start when I leave my house, get to California, enter the park, or get on a ride? When does it end?

Eventually, Lynn and Nancy suggested the best estimate is no estimate at all.

Instead, it is a negotiation with the business (or your manager). When someone asks, “how long will testing take?”, maybe you should explain that testing is not a phase. Testing is exploration, discovery, learning, and reporting. Testing could end when there are no more interesting questions to answer but stopping testing is a business decision.

They further suggested that testers have a responsibility to help the business understand the trade-offs; if quality expectations are high, it may require more testing than if they are lower. Change your test approach to fit the needs. If you only have one day to test, you can still do your best to find the most mission critical information that you can in one day.

Apart from the session content, Lynn and Nancy were awesome presenters. They used volunteers from the audience for role plays, cartoons, brainstorms, and other interactive techniques to keep the session engaging. I heard they proposed a full day session for STPCon Spring. That would be a fun day.

Thanks Nancy and Lynn!


  1. Keith Rozario said...

    That's an excellent piece.

    I've always been fascinated with testing and even more fascinated with the approach people take to testing. To some testing is a matter of ticking check boxes, to check that the developers delivered as promised without bugs. Since it's just ticking the boxes, estimating how much time it will take shouldn't prove too much of a problem.

    Using a top down approach, where the scope of testing is defined before the timeline, rather than a bottom up approach , where the timeline is defined before the scope is a matter that differs from project to project, but no matter which approach you take if you think of testing as a exploratory adventure of discovery rather than a mandatory ticking of boxes.....then the battle is already half won :)

  2. Anonymous said...

    I like to ask from applying Testers an open question like "when the testing should end?". Answers are telling so much of persons skills/mindset/seniority/business and project understanding

Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.
No part of the content or the blog may be reproduced without prior written permission.