I think it’s only people who experience bugs.
Sadly, devs, BAs, other testers, stakeholders, QA managers, directors, etc. seldom appear interested in the fruits of our labor. The big exception is when any of these people experience a bug, downstream of our test efforts.
“Hey, did you test this? Did it pass? It’s not working when I try it.”
Despite the disinterest, us testers spend a lot of effort standing up ways to report test results. Whether it be elaborate pass/fail charts or low-tech information-radiators on public whiteboards, we do our best. I’ve put lots of energy into coaching my testers to give better test reports but I often second guess this…wondering how beneficial the skill is.
Why isn’t anyone listening? These are some reasons I can think of:
- Testers have done such a poor job of communicating test results, in the past, that people don’t find the results valuable.
- Testers have done such a poor job of testing, that people don’t find the results valuable.
- People are mainly interested in completing their own work. They assume all is well with their product until a bug report shows up.
- Testing is really difficult to summarize. Testers haven't found an effective way of doing this.
- Testing is really difficult to summarize. Potentially interested parties don’t want to take the time to understand the results.
- People think testers are quality cops instead of quality investigators; People will wait for the cops to knock on their door to deliver bad news.
- Everyone else did their own testing and already know the results.
- Test results aren’t important. They have no apparent bearing on success or failure of a product.
We had a relatively disastrous prod deployment last week. Four bugs, caused by a large refactor, were missed in test. But here’s the weirder part, along with those four bugs, the users started reporting previously existing functionality as new bugs, and in some cases, convincing us to do emergency patches to change said previously existing functionality.
It seems bugs beget bugs.
Apparently the shock of these initial four bugs created a priming effect, which resulted in overly-critical user perceptions:
“I’ve never noticed that before…must be something else those clowns broke.”
I’ve heard people are more likely to tidy up if they smell a faint scent of cleaning liquid. Same thing occurs with bugs I guess.
What’s the lesson here? Releasing four bugs might be more expensive than fixing four bugs. It might mean fixing seven and dealing with extra support calls until the priming effect wears off.