We all have different versions of our product on different environments, right? For example: If Iteration 10 is in Production, Iteration 11 is in one or more QA environments. When bugs exist in both Iterations, we have BIMIs (Bugs-In-Multiple-Iterations).
One of my project teams just found a gap in our process that resulted in a BIMI hitching a ride all the way to production. That means our users found a bug, we fixed it, and then our users found the same bug four weeks later (and then we fixed it again!). Our process for handling bugs had always been to log one bug report per bug. Here is the problem.
- Let’s say we have a production bug (in Iteration 10).
- Said bug gets a bug report logged.
- Our bug gets fixed and tested in our post-production environment (i.e., test environment with same bits as production).
- Finally, the fix deploys to production and all is well, right? The bug report status changes to “Closed”.
- Now we can get back to testing Iteration 11.
What did we forget?
…well it’s probably not clear from my narrative but our process gap is that the above bug fix code never got deployed to Iteration 11 and the testers didn’t test for it because “Closed” bugs are already fixed in prod, and thus, off the testers’ radar.
If our product was feasible to automate, we could have added a new test to our automation suite to cover this. But in our mostly manual process, we have to remember to test for BIMIs. The fact is, the same bug fix can be Verified in one iteration and Failed in another. The bug can take a different path in each environment or, like in my case, fail to deploy to the expected environment at all.
This iteration we are experimenting with a solution. For BIMIs, we are making a separate copy of the bug report and calling it a “clone”. This may fly in the face of leaner documentation teams but we think it’s a good idea based on our history.
What’s your solution for making sure bug fixes make it into the next build?