As testers, we typically open screens in our products and focus on the data that is presented to us.  Does it meet our expectations?  We are sometimes forced to be patient and wait for said data to present itself.  Don’t forget to step back and determine what happens if the data doesn’t get a chance to present itself.

Test Steps:

1. Open a screen or window on the UI.  This bug is easier to repro in places where lots of data needs to load on the screen or window.  Look for asynchronous data loading (e.g., the user is given control after some data presents itself, while the rest of the data is still being fetched)

2. Close the screen or window before the data has finished loading.  Be quick.  Be creative; if the screen you opened doesn’t close on command, try its parent.

Expected Results:  No errors are thrown.  The product handles it gracefully by killing the data load process and releasing the memory.

After reading my Testing for Bug Bucks & Breaking The Rules post, In2v asked, “I was wondering if your team went on with that game and how it turned out. Would you update us?”

Sure.  The Bug Bucks game/experiment failed miserably.  It sounded good on paper but in practice it was too awkward.  When testers found bugs, they didn’t have the heart to ask Progs for Bug Bucks.  Go figure…

We like to joke about testers and Progs being caught up in some kind of rivalry. But when we’re actually performing our jobs on serious work, we want each other to succeed.  Both tester and Prog had an understanding that nobody was at fault; they were both doing their jobs.  When I asked TesterA why he didn’t take any Bug Bucks from ProgA, I was told ProgA would have been in Bug Buck debt (because of all the bugs) but it didn’t seem fair because many of the bugs were team oversights.

We could have adjusted the rules but I could see it was too awkward.   And just before management was getting ready to order some Bug Buck goods for purchase, I suggested we pull the plug.

So now I have a stack of worthless Bug Bucks on my desk, and a recurring hope to come up with a more fun way to perform our jobs.  I’m open to suggestions…

From this post forward, I will attempt to use the term “Prog” to refer to programmers.

I read a lot of Michael Bolton and I agree, testers are developers too.  So are the BAs.  Testers, Programmers, and BAs all work together to develop the product.  We all work on a software development team. 

Before I understood the above, I used the word “Dev” as short hand for “developer” (meaning programmer).  Now everybody on my development team says “Dev” (to reference programmers).  It has been a struggle to change my team culture to get everyone to call them “programmers”.  I’m completely failing and almost ready to switch back to “Dev”, myself.  I don’t much like the word, “programmer”…too many syllables and letters.

Thus, I give it one last attempt.  “Prog” is perfect!  Please help me popularize this term. It’s clearly short for “programmer”, easy to spell, and fun to say.  It also reminds me of “frog”, which is fitting because some progs are like frogs.  They sit all day waiting for us to give them bugs.

We have a recurring conversation on both my project teams.  Some Testers, Programmers, BAs believe certain work items are “testable” while others are not.  For example, some testers believe a service is not “testable” until its UI component is complete.  I’m sure most readers of this blog would disagree.

A more extreme example of a work item, believed by some to not be “testable”, is a work item for Programmer_A to review Programmer_B’s code.  However, there are several ways to test that, right?

• Ask Programmer_A if they reviewed Programmer_B’s code.  Did they find problems?  Did they make suggestions?  Did Programmer_B follow coding standards?
• Attend the review session.
• Install tracking software on Programmer_A’s PC that programmatically determines if said code was opened and navigated appropriately for a human to review.
• Ask Programmer_B what feedback they received from Programmer_A.

IMO, everything is testable to some extent.  But that doesn’t mean everything should be tested.  These are two completely different things.  I don’t test everything I can test.  I test everything I should test.

I firmly believe a skilled tester should have the freedom to decide which things they will spend time testing and when.  In some cases it may make more sense to wait and test the service indirectly via the UI.  In some cases it may make sense to verify that a programmer code review has occurred.  But said decision should be made by the tester based on their available time and queue of other things to test.

We don’t need no stinkin’ “testable” flag.  Everything is testable.  Trust the tester.

Hey Testers, let’s start paying more attention to our bug language.  If we start speaking properly, maybe the rest of the team will join in.

Bug vs. Bug Report:

We can start by noting the distinction between a bug and a bug report.  When someone on the team says, “go write a bug for this”, what they really mean is “go write a bug report for this”.  Right?  They are NOT requesting that someone open the source code and actually write a logic error.

Bug vs. Bug Fix:

“Did you release the bug?”.  They are either asking “did you release the actual bug to some environment?” or “did you release the bug fix?”.

Missing Context:

“Did you finish the bug?”.  I hear this frequently.  It could mean “did you finish fixing the bug?” or it could mean “did you finish logging the bug report?” or it could mean “did you finish testing the bug fix?”.

Bug State Ambiguity:

“I tested the bug”.  Normally this means “I tested the bug fix.”  However, sometimes it means “I reproduced the bug.”…as in “I tested to see if the bug still occurs”.

It only takes an instant to tack the word “fix” or “report” onto the word “bug”.  Give it a try.

A fun and proud moment for me.  Respected tester, Matt Heusser, interviewed me for his This-Week-In-Software-Testing podcast on Software Test Professionals.  It was scary because there was no [Backspace] key to erase anything I wished I hadn’t said.

I talked a bit about the transition from tester to test manager, what inspires testers, and some other stuff.  It was truly an honor for me.

The four most recent podcasts are available free, although you may have to register for a basic (free) account.  However, I highly recommend buying the $100 membership to unlock all 49 (and counting) of these excellent podcasts.  I complained at first but after hearing Matt’s interviews with James Bach, Jerry Weinberg, Cem Kaner, and all the other great tester/thinkers, it was money well spent.  The production is top notch and listening to Matt’s testing ramblings on each episode is usually as interesting as the interview.  There are no podcasts available like these anywhere.

Keep up the great work Matt and team!  And keep the podcasts coming!

We all have different versions of our product on different environments, right?  For example: If Iteration 10 is in Production, Iteration 11 is in one or more QA environments. When bugs exist in both Iterations, we have BIMIs (Bugs-In-Multiple-Iterations).

One of my project teams just found a gap in our process that resulted in a BIMI hitching a ride all the way to production.  That means our users found a bug, we fixed it, and then our users found the same bug four weeks later (and then we fixed it again!).  Our process for handling bugs had always been to log one bug report per bug.  Here is the problem. 

1. Let’s say we have a production bug (in Iteration 10).
2. Said bug gets a bug report logged.
3. Our bug gets fixed and tested in our post-production environment (i.e., test environment with same bits as production).
4. Finally, the fix deploys to production and all is well, right?  The bug report status changes to “Closed”.
5. Now we can get back to testing Iteration 11.

What did we forget?

…well it’s probably not clear from my narrative but our process gap is that the above bug fix code never got deployed to Iteration 11 and the testers didn’t test for it because “Closed” bugs are already fixed in prod, and thus, off the testers’ radar.

If our product was feasible to automate, we could have added a new test to our automation suite to cover this.  But in our mostly manual process, we have to remember to test for BIMIs.  The fact is, the same bug fix can be Verified in one iteration and Failed in another.  The bug can take a different path in each environment or, like in my case, fail to deploy to the expected environment at all.

This iteration we are experimenting with a solution.  For BIMIs, we are making a separate copy of the bug report and calling it a “clone”.  This may fly in the face of leaner documentation teams but we think it’s a good idea based on our history.

What’s your solution for making sure bug fixes make it into the next build?