Most bug readers will agree, a simple “Expected Results” vs. “Actual Results” statement in the bug description will remove all ambiguity. But what about the bug title? Is a bug title supposed to include the expected results, actual results, or both? Every time I log a bug, I pause to consider the various title possibilities. I want a unique but concise title that will summarize the bug, making the description as unnecessary as possible. My head swims with choices…

• If user does X after midnight, user gets Y.
• If user does X after midnight, user should not get Y.
• If user does X after midnight, user should get Z.
• If user does X after midnight, user gets Y instead of ZUser should get Z when they do X after midnight.
• etc…

Here is what I think.

There is no “best” bug title. Any bug title is good enough as long as it:

• is unique within my bug tracking system
• describes the problem to some extent
• includes key words (these words will be used someday to find this bug; searching for bugs where title contains some text)

So unless someone convinces me otherwise, with a comment on this post, I have decided to just use the first distinct bug title popping into my mind and stop worrying about bug title perfection.

Roshni Prince asks,

Can you suggest some tips to deal with testers that attempt to dig into code and fix their own bugs?

I like the question, so I’ll tell you what I think.

If we had unlimited test time, SmartyPantsTester would rock! However, if we have to provide as much information as possible, about the current AUT quality, in a limited time, SmartyPantsTester gets in our way.

So how do we deal with SmartyPantsTester?

The Carrot Approach:

Convince SmartyPantsTester the real hero is the tester who can tell us something meaningful about the AUT quality. Can anyone help us…Please? We need someone smart enough to find the weak points in our AUT? We need someone familiar enough with the business to tell us if FeatureA will solve the user’s problems. Is anyone creative enough to determine how to test the feature the devs said was impossible to test? Is anyone methodical enough to determine the repro steps to this intermittent problem? We need someone brave enough to QA Certify this AUT for production. Get it? Appeal to the ego.

The Stick Approach:

Ask SmartyPantsTester to work extra hours until she can answer questions like the following:

• Cool, you found an incorrect join statement, how does the rest of the AUT look?
• Do the new features work properly?
• How much of the AUT have you tested?
• How many tests have you executed?
• How many showstopper bugs have you logged?
• In your opinion, is the AUT ready to ship?

And once again, I find myself with the same conclusion; there is simply too much to test in the available time. Testers reduce their chances of success by trying to do the devs’ job too.

Should you log a bug without knowing the expected results?

Someone chuckled at my “?” for the expected results, during a bug review meeting. The bug (like many of my bugs) looked like this:

Repro Steps:
1. do something
2. do something else
3. do something else

Expected Results: ?
Actual Results: an unhandled error is thrown

Good testers determine scenarios that nobody thought of. That is one skill that makes us good testers. Some time ago, I didn’t let myself log bugs until I tracked down the proper oracles and determined the expected results; a practice that sounds good…until you try it. Unfortunately, the oracles are never quick to respond. Often they pose questions to the users, which open additional discussions, meetings, etc…until the tester forgets the details of their wonderful test.

So these days, if I encounter a bug and I don’t know the expected results, I log the bug immediately and let someone else worry about expected results…if they even care. It’s not because I’m too lazy to seek out my oracles. It’s because my time is better spent logging more bugs! When in doubt, remember, the tester’s primary job is to provide information about the AUT. It’s not the responsibility of the tester to determine expected results. If the tester identifies a scenario that will crash the AUT, they should log the bug now.

I’m surprised by how many people still send around bmp files of their entire desktop when they are only interested in showing some small error message displaying in a little window. They are using the [Print Screen] key. Some, at least know they can use [Alt]/[Print Screen] to capture only the active window.

Others prefer to capture only the area the audience needs to understand; they may use a screen capture app. I’ve been using Wisdom-soft’s free ScreenHunter . I’ve got it customized to capture the area within a rectangle I draw, after pressing F6. After drawing my rectangle, its contents capture as an auto-named gif file and a clipboard item.

Screen-capture-type-stuff I think about :

• I try to avoid screen capturing error messages, opting instead to capture the error message in text format, from an error log. That way the dev can see the whole message and copy the text if they want to search the code or something. If the devs don't log the error, they're stuck with a screen capture.
• If the screen capture needs other context (e.g., which programs are running in my tray, what time is it) I still capture the entire desktop.
• Occasionally I mark up the screen capture (in Paint.NET) to circle something or add other annotations.
• If capturing action is better, I capture video.
• Sometimes I save time by using a screen capture to support repro steps. Example: Capture a filter page for a report and write a repro step that says "specify all filter criteria as depicted in the screen capture".
  

What (free) screen capture program do you use? What screen capture tips did I miss?

Chances are, your AUT probably has buttons on the UI somewhere. If so, those buttons trigger actions. A common oversight is to not handle multiple actions being triggered at nearly the same time.

Testers and devs are familiar with standard UI controls. We know buttons don’t generally require double-clicks. However, many users don’t have this instinct. These users double-click everything, even buttons. Become one of them!

My AUT had a bug that actually allowed users to rapid-fire-click a generate invoice button, and get duplicate invoices. Ah…yikes.

Here is the bread and butter test:
Get your mouse over a button that triggers an action. Get your finger ready. Click that button multiple times as quickly as you can. Now go look in the DB, error logs, or wherever you need to look to determine if multiple actions where triggered inappropriately. No bug? Try it a few more times. Or try getting the focus on the button and using a rapid-fire [Enter] key.

Got any variations on this?

The more one learns about the inner workings of an AUT, the more one may get distracted from logging important bugs.

I saw this happen last week with a fellow teammate. She discovered a search criteria control that didn’t allow the user to search by certain values (e.g., they could search by A, B, D, but not C). Instead of logging a bug, she explained to me why it didn’t work. She was thrilled with her knowledge of countless dev discussions trying to fix the data structure beneath said search control. It was more exciting for her to explain the complex data than to worry about the little users who would not be able to search per their expectations. It was like she was suddenly on the side of the developers…and the users would never understand the complex challenges of making software work. “It’s not a bug, nobody can figure out how to deal with the complex data”.

Huh?

Dumb testers don’t have this problem. If it doesn’t follow the spec, they log it. And that’s the good thing about dumb testers. Be careful with your knowledge.

Alex and Joe,

I agree with both your comments to my last post.

Joe blames the dev…

“- No bug report- [dev] corrected it on their own”

Obviously devs catch defects in their code (logic errors, missed specs, inefficiencies). If devs find and resolve defects in previously released code, should a bug be logged? On one hand, if the dev corrects it before anyone else catches it, why should the dev have to tell anyone? If a tree falls in the forest and nobody is around, does it make a sound? On the other hand, we can’t assume an unnoticed defect in the field, means said defect’s resolution will also be unnoticed. Thus, IMO it’s best for devs to let the team know about all changes. Typical ways to do this:

• Dev logs bug


• Dev tells tester to log bug


• Dev logs a work item representing work they did (e.g., refactored code for FeatureA). The work item should be visible to the team. The dev gets credit for their wonderful coding, and the tester gets the tap on the shoulder to attempt some testing.

Alex blames the tester…

“That test should have failed a long time ago.”

In this case, the feature did not match its spec. You're damn right, the tester should have caught this! The tester should thank their dev for doing the testers job. (Unfortunately, if the tester relied on the spec, we would be right back where we started.)

So a third member must also take the blame; the spec author (e.g., Business Analyst), who did not capture the true business need of the feature. However, we can’t put all the blame on them. “The requirements sucked!” …this is a tired argument, one that I decided never to waste my time with. Anyone who disagrees probably has not attempted to write requirements.

The next time you feel compelled to point the finger at a teammate, don’t. Responsibilities blur across the software development trades and the whole team can share success or failure.

(sorry, this post is a little dull. I’ll try for something juicier next week)