I finally added Perlclip to my tray. I use it several times a week when I have to test text field inputs on my AUT. Among other things, this helpful little tool, created by James Bach, allows one to generate a text string with a specific character count. That alone is not very cool. However, the fact that the generated text string is comprised of numbers that indicate the count of the following astrisk is way cool.

Example: If I create a "counterstring" of 256 characters, I get a string on my clipboard that can be pasted. The last portion of the string looks like this...

12*216*220*224*228*232*236*240*244*248*252*256*

Each number is telling you the character number of its following astrisk. Thus, the last astrisk is character #256. The last "6" is character #255. Get it? So if you don't have a boundary requirement for a text input field, just paste in something huge and examine the string that got saved. If the last portion of the saved string looks like this...

0*43*46*49*52*55*58*61*6

...your AUT only accepted the first 62 characters.

The first Agile Manifesto value is…

“Individuals and interactions over processes and tools”

While reading this recently, something radical occurred to me. If I practice said value to its fullest, I can stop worrying about how to document everything I test, and start relying on my own memory.

I hate writing test cases. But the best argument for them is to track what has been tested. If someone (or myself) asks me “Did you test x and y with z?”, a test case with an execution result seems like the best way to determine the answer. However, in practice, it is not usually the way I answer. The way I usually answer is based on my memory.

And that, my friends, is my breakthrough. Maybe it’s actually okay to depend on your memory to know what you have tested instead of a tool (e.g., Test Director). But no tester could ever remember all the details of prior test executions, right? True, but no tester could ever document all the details of prior test executions either, right? To tip the balance in favor of my memory being superior to my documentation skills, let me point out that my memory is free. It takes no extra time like documentation does. That means, instead of documenting test cases, I can be executing more test cases! And even if I did have all my test case executions documented, sometimes it is quicker to just execute the test on the fly than go hunt down the results of the previous run (quicker and more accurate).

It all seems so wonderful. Now if I can figure out how to use my memory for SOX compliancy... shucks.

Before we test, we should plan. Before we plan, we should understand. Before we understand, we should discuss. As we discuss complex paths through software, we should use models.

As soon as you draw a model, the discussion becomes engaging for both parties. Each person is forced to pay attention because the items under discussion are now concrete entities we can point to. Our brains can focus less on what the names or actions for these objects are and more on how they interact.

Even the crudest models (e.g., circles and arrows) will facilitate understanding. Everyone wants to discuss things with a model, whether they know it or not. It’s easier to talk to the model than someone’s face, especially for us introverts. When I sense confusion during a discussion I say “let’s go the white board”. Once the other person (usually the dev) gets their butt out of their chair the rest is down hill.

My knowledge of the inner workings of my AUT is miniscule compared with that of my devs and I’m not afraid to show it. I’ll draw some first grader shapes with letters and before I know it the dev is reaching for the dry erase marker to show me how it really works. And the beauty of the whole thing is that your brain will use the model to develop test cases; the possible inputs/outputs become clearer. Say these out loud and your dev will help you find the weak points. Then thank the dev and go getcha some bugs!

My first job out of college was teaching software applications like MS Access, FoxPro, and Act!. Back then, in the late 90's, demand for these types of classes was much higher than it is today. This, I believe, is because today's software users are more sophisticated. Most have already been exposed to some flavor of word processing, spreadsheet, or email applications. Many can even teach themselves software or look online for answers.

After accepting the above, it's not too great a leap to also accept that modern software users are aware that software is not perfect. They have experienced application hangs and strange system errors and many users learn to avoid these bugs or recover via a reboot or similar.

If the above is true, why can't all bug lists be public? The culture of my dev team prefers to keep the bug list hidden because they believe users will have trust issues if we admit to known production bugs. I disagree. In fact, if properly facilitated, I think a public bug list can actually build user trust. Users are smart enough to see the value in having their software earlier, even at the expense of known bugs.

What do you think?

Contrary to my previous post, about devs taking more blame for production bugs, devs also take most of the credit when users like an application. I’ll bet testers rarely get praise from end users.

The reason is simple. Users don’t read the fixed bug list. Users have no idea how crappy the app was before the testers started working their magic. Have you ever heard a user say, “Wow, these testers really worked hard and found a lot of bugs!”. Users don’t have this information. For all they know, testers didn’t do a damn thing. The Devs could have written rock solid code and tested it themselves…who knows?

I’ve had the luxury of working on an AUT that hasn’t gone live for 3 years. Now that we’re live, the old familiar tester stress, guilt, and anger is back.

When the first major production bug was discovered I wanted to throw up. I felt horrible. Several people had to work around the clock to clean up corrupt data and patch the problem. I wanted to personally apologize, to each person on my team and hundreds of users, for not catching the bug in testing…and I did apologize to a couple individuals and offer my help. Apologizes in these cases don’t help at all, other than for personal guilt and accountability.

During my selfish guilt, I opened my eyes and realized my fellow devs felt just as accountable as I did (if not more so), and never attempted to pass blame to me. I started asking myself who is really more at fault here; the tester who didn’t test the scenario or the developer who didn’t code to handle it?

I think the tester is 75% responsible for the bug and the developer, 25%. However, the dev probably gets the brunt of the blame because they are a more prominent part of the development team. I would guess more end users have heard of people called software developers than have heard of people called software testers.