The Fog Of Test

Test Manager: Remember, we're Software Testers, not some sorry-ass QA Analysts. We're elite. Let's act like it out there. Hoo-ah?

Testers: Hoo-ah!

You arm yourself with TestA and prepare to battle your AUT.

Take a deep breath, head into the AUT, and begin executing TestA. Prior to observing your expected results, you determine TestA is blocked from further execution (call it BlockageA). You make a note to investigate BlockageA later. You modify TestA slightly to give it a workaround in an attempt to avoid BlockageA. TestA encounters BlockageB. Now you decide to deal with BlockageB because you are out of workarounds. Is BlockageB a bug? You can’t find the specs related to BlockageB. After an hour, your BA finds the specs and you determine BlockageB is a bug (BugB). You check the bug DB to see if this bug has already been logged. You search the bug DB and find BugC, which is eerily similar but has different repro steps than your BugB. Not wanting to log dupe bugs you perform tests related to BugB and BugC to determine if they are the same. Finally you decide to log your new bug, BugB. One week later BugB gets rejected because it was “by design”; the BA forgot to update the feature but verbally discussed it with dev. Meanwhile, you log a bug for BlockageA and notice four other potential problems while doing so. These four potential problems are lost because you forgot to write a follow-up reminder note to yourself. Weeks later BlockageA is fixed. You somehow stayed organized enough to know TestA can finally be executed. You execute TestA and it fails. You log BugD. BugD is rejected because TestA’s feature got moved to a future build but dev forgot to tell you. Months later, TestA is up for execution again. TestA fails and you log BugE. The dev can’t repro BugE because their dev environment is inadequate for testing. Dev asks tester to repro BugE. BugE does not repro because you missed an important repro step. Now you are back at the beginning.

You’ve just experienced the "fog of test".

The "fog of test" is a term used to describe the level of ambiguity in situational awareness experienced by participants in testing operations. The term seeks to capture the uncertainty regarding own capability, AUT capability and stakeholder intent during an engagement or test cycle. (A little twist on the “Fog of war” Wikipedia entry)

4 comments:

  1. AngryElf said...

    In the gaming world we have a different term for this: Epic Fail

  2. Michael Kelly said...

    Mmmmm.... Black Hawk Down....

  3. Eric Jacobson said...

    Michael,

    Well done. You nailed it. Great movie, huh? A day testing my AUT is like a day in Mogadishu Somalia trying to weed out rebel forces.

  4. Anonymous said...

    that was great !



Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.
No part of the content or the blog may be reproduced without prior written permission.