See Part 1 for intro.
- There are two reasons why your bugs are not getting fixed:
- There is more important stuff going on
- You are not explaining them well.
- The tester didn’t think of it.
- The tester thought of it but decided not to execute it. Hopefully it’s the latter. It may be worth while to brainstorm on tests.
- Exploratory behavior = Value seeking.
- Scripted behavior = Task seeking
- A story about the product (e.g., is the product any good?).
- A story about how you tested it (e.g., how do I know? Because I tested it by doing this…).
- A story about the value of the testing (e.g., here is why you should be pleased with my work…).
Rapid Software Testing (RST) is for situations where you have to test a product right now, under conditions of uncertainty, in a way that stands up to scrutiny.
I don’t want to walk you through the exercises, videos, and discussions Michael Bolton used in class because…well, it’s his class, and you should take it! But I will share some bite-sized test wisdom I wrote in my notebook during class.
Be careful, most of these are heuristic…
- An assumption is the opposite of a test. (I love that!)
- Our job is not to be “done” with something. It’s to find out interesting things that others do not already know.
- A tester’s job is to see the complexity behind the seemingly simple and to see the simplicity behind the seemingly complex.
- “Test” is a verb. Not a noun. It’s not an artifact. It’s something one does.
- Testers do not put the quality in, they help others put the quality in. Testers do not assure quality, but if we must use the term “QA”, maybe it should stand for Quality Assistance.
- Testers are like editors for writers. No matter how well a writer tests their work, a good editor can normally find mistakes.
- Programmers do lots of testing. But they need help finding the last few problems.
- A tester’s job is to remain uncertain, to maintain the idea that it might not work.
- providing a “QA Certification” is like your manager making you say “I will take the blame…”
- Testers don’t matter because the program is not intended for them.
- Discussions about quality are always political or emotional. – Jerry Weinberg
- An “Issue” is anything that threatens the value of our testing. Issues should be reported. They may be more important than bugs because they give bugs time to hide.
- Threats to testability are “issues”. Two things that threaten testability are:
- Visibility (e.g., log files)
- Controllability – the capacity to make the program do stuff (e.g., can I update the DB and config files?)
- “Positive Test” – Fulfills every required assumption. Entering a bad password is a positive test if we’ve already established how the bad password should be handled.
- What is testing? Getting answers.
- A useful test approach:
- Know your mission
- Consider building a model of the product first
- Begin sympathetically
- Then chase risks
- The first few moments of a product should be based on learning.
- There’s always more than meets the eye.
- Maps and models that you build don’t have to be right. They just need to get people thinking.
- If you don’t have enough time to test, one trick to get more time is to find important bugs. People will generally delay releases. (But don’t sit on bugs until the last minute, of course. Report them as soon as you’re aware.)
- Don’t forget to “imploy the pause”. Take the time to learn something new every now and then.
Yes, Michael Bolton is one of my biggest mentors. And you’ve read a lot of fanboy posts on this blog. But before I start spewing stuff from my RST notes, I want to post a disagreement I had with Michael Bolton (and RST). After a 15 minute discussion, he weakened my position. But I still disagree with this statement:
We don’t test to find out if something works. We test to find out if it doesn’t work.
Here is a reason I disagree: Knowing at least one way software can work, may be more valuable than knowing a thousand ways it can NOT work.
Example: Your product needs to help users cross a river. Which is more valuable to your users?
- “hey users, if you step on these exact rocks, you have a good chance of successfully crossing the river”
- “hey users, here are a bunch of ways you can NOT cross the river: jump across, swim past the alligators, use the old rickety bridge, swing across on a vine, drain the river, dig a tunnel under it, etc.”
Users only need it to work one way. And if it solves a big enough problem, IMO, those users will walk across the rocks.
Sure, finding the problems is important too. Really important! But if someone puts a gun to my head, and says I only get one test. It’s going to be a happy path test.
Bolton referred us to the following discussion between James Bach and Michael Kelly (http://michaeldkelly.com/media/ then click on “Is there a problem here?”). I thought it would change my mind, as most James Bach lessons do. It hasn’t…yet.
I might be wrong.
I finally pulled it off! My company brought Michael Bolton to teach a private 3-day Rapid Software Testing course and stick around for a 4th day of workshops and consulting. On the fourth day I had Michael meet with QA Managers to give his talk/discussion on “How to Get The Best Value From Testing”. Then he gave a talk for programmers, BAs, testers, and managers on “The Metrics Minefield”. Finally, he did a 2.5 hour workshop on “Critical Thinking for Testers”.
My brain and pen were going the whole four days; every other sentence he uttered held some bit of testing wisdom. I’ll post chunks of it in the near future. I attended the class 6 years earlier in Toronto and I was concerned it would have the same material but fortunately most of it had changed.
The conversations before/after class were a real treat too. After the first day, Claire Moss, Alex Kell, Michael Bolton, and I met at Fado for some Guinness, tester craic, and much to my surprise, to listen to Michael play mandolin in an Irish tradition music session. He happened to be a very good musician and (of course) gave us handles to tell a slip jig from a reel.
Several days later, I’m still haunted by Michael-Bolton-speak. I keep starting all my sentences with “it seems to me”. But best of all perhaps, is the lingering inspiration to read, challenge, and contribute thoughtful ideas to our testing craft. He got me charged up enough to love testing for at least another 6 years. Thanks, Michael!