After RST class (see my Four Day With Michael Bolton post), Bolton did a short critical thinking for testers workshop.  If you get an opportunity to attend one of these at a conference or other place, it’s time well spent.  The exercises were great, but I won’t blog about them because I don’t want to give them away.  Here is what I found in my notes…

  • There are two types of thinking:
    1. System 1 Thinking – You use it all the time to make quick answers.  It works fine as long as things are not complex.
    2. System 2 Thinking – This thinking is lazy, you have to wake it up.
  • If you want to be excellent at testing, you need to use System 2 Thinking.  Testing is not a straight forward technical problem because we are creating stuff that is largely invisible.
  • Don’t plan or execute tests until you obtain context about the test mission.
  • Leaping to assumptions carries risk.  Don’t build a network of assumptions.
  • Avoid assumptions when:
    • critical things depend on it
    • when the assumption is unlikely to be true
    • the assumption is dangerous when not declared
  • Huh?  Really?  So?   (James Bach’s critical thinking heuristic)
    • Huh? – Do I really understand?
    • Really? – How do I know what you say is true?
    • So? – Is that the only solution?
  • “Rule of Three” – If you haven't thought of at least three plausible explanations, you’re not thinking critically enough.
  • Verbal Heuristics: Words to help you think critically and/or dig up hidden assumptions.
  • Mary Had a Little Lamb Heuristic – emphasize each word in that phrase and see where it takes you.
  • Change “the” to “a” Heuristic:
    • “the killer bug” vs. “a killer bug”
    • “the deadline” vs. “a deadline”
  • “Unless” Heuristic:  I’m done testing unless…you have other ideas
  • “Except” Heuristic:  Every test must have expected results except those we have no idea what to expect from.
  • “So Far” Heuristic:  I’m not aware of any problems…so far
  • “Yet” Heuristic: Repeatable tests are fundamentally more valuable, yet they never seem to find bugs.
  • “Compared to what?” Heuristic: Repeatable tests are fundamentally more valuable…compared to what?
  • A tester’s job is to preserve uncertainty when everyone around us is certain.
  • “Safety Language” is a precise way of speaking which differentiates between observation and inference.  Safety Language is a strong trigger for critical thinking.
    • “You may be right” is a great way to end an argument.
    • “It seems to me” is a great way to begin an observation.
    • Instead of “you should do this” try “you may want to do this”.
    • Instead of “it works” try “it meets the requirements to some degree”
    • All the verbal heuristics above can help us speak precisely.


  1. Michael Bolton said...

    Hi, Eric...

    First, I'm very grateful for your kind words. Thank you!

    Second, the System 1 / System 2 ideas come to me from Daniel Kahneman and his book Thinking Fast and Slow, which is a terrific addition a tester's reading list. Highly recommended.


    ---Michael B.

Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.
No part of the content or the blog may be reproduced without prior written permission.