Which of the above questions is more important for testers to ask?
Let’s say you are an is-there-a-problem-here tester:
- This calculator app works flawlessly as far as I can tell. We’ve tested everything we can think of that might not work and everything we can think of that might work. There appear to be no bugs. Is there a problem here? No.
- This mileage tracker app crashes under a load of 1000 users. Is there a problem here? Yes.
But might the is-there-a-problem-here question get us into trouble sometimes?
- This calculator app works flawlessly…but we actually needed a contact list app.
- This mileage tracker app crashes under a load of 1000 users but only 1 user will use it.
Or perhaps the is-there-a-problem-here question only fails us when we use too narrow an interpretation:
- Not meeting our needs, is a problem. Is there a problem here? Yes. We developed the wrong product, a big problem.
- A product that crashes under a load of 1000 users may actually not be a problem if we only need to support 1 user. Is there a problem here? No.
Both are excellent questions. For me, the will-it-meet-our-needs question is easier to apply and I have a slight bias towards it. I’ll use them both for balance.
Note: The “Will it meet our needs?” question came to me from a nice Pete Walen article. The “Is there a problem here?” came to me via Michael Bolton.