Boycott the International Institute for Software Testing
1 comments Published by Eric Jacobson on Jul 19, 2008 at 9:20 AMI first attended one of IIST's international testing certification weeks five years ago. The certification requires certain classes, some of which are taught by Dr. Magdy Hanna. After taking his course, The Principles of Software Testing, I gave him a poor evaluation. I may be one of the few people in the world who actually bothers to provide valuable feedback on surveys and I'm not shy about giving it.
Anyway, two years later I talked my new company into letting me attend another certification week. I was only about three courses away from certification. I registered, signed up for the courses I needed, and bought an airline ticket. About a week later, I got an email saying I could not take one of the courses I signed up for because the instructor would not allow it (due to a poor evaluation I had given him two years earlier).
Was this guy for real? Dr. Magdy Hanna would not return my calls but eventually sent me an email saying since I didn't like his teaching style I would be banned from his classes. Oh, I would have taken someone else's class in a heartbeat but this was the last one I needed and Hanna happened to be the only guy teaching it that week. It would have been nice if they would have told me prior to me registering and buying an airline ticket, which I had to use elsewhere.
Anyway, I hate IIST. Hanna's classes did suck and so did his teaching technique, which was more about his own ego than trying to help anyone become a better tester. That year I attended Michael Bolton's Rapid Software Testing course and learned more than I had learned in all the IIST classes. This year I am attending the Software Test & Performance Conference and next year I hope to try CAST. If you ever have a choice, don't choose IIST.
My first job out of college was teaching software applications like MS Access, FoxPro, and Act!. Back then, in the late 90's, demand for these types of classes was much higher than it is today. This, I believe, is because today's software users are more sophisticated. Most have already been exposed to some flavor of word processing, spreadsheet, or email applications. Many can even teach themselves software or look online for answers.
After accepting the above, it's not too great a leap to also accept that modern software users are aware that software is not perfect. They have experienced application hangs and strange system errors and many users learn to avoid these bugs or recover via a reboot or similar.
If the above is true, why can't all bug lists be public? The culture of my dev team prefers to keep the bug list hidden because they believe users will have trust issues if we admit to known production bugs. I disagree. In fact, if properly facilitated, I think a public bug list can actually build user trust. Users are smart enough to see the value in having their software earlier, even at the expense of known bugs.
What do you think?
Contrary to my previous post, about devs taking more blame for production bugs, devs also take most of the credit when users like an application. I’ll bet testers rarely get praise from end users.
The reason is simple. Users don’t read the fixed bug list. Users have no idea how crappy the app was before the testers started working their magic. Have you ever heard a user say, “Wow, these testers really worked hard and found a lot of bugs!”. Users don’t have this information. For all they know, testers didn’t do a damn thing. The Devs could have written rock solid code and tested it themselves…who knows?
I’ve had the luxury of working on an AUT that hasn’t gone live for 3 years. Now that we’re live, the old familiar tester stress, guilt, and anger is back.
When the first major production bug was discovered I wanted to throw up. I felt horrible. Several people had to work around the clock to clean up corrupt data and patch the problem. I wanted to personally apologize, to each person on my team and hundreds of users, for not catching the bug in testing…and I did apologize to a couple individuals and offer my help. Apologizes in these cases don’t help at all, other than for personal guilt and accountability.
During my selfish guilt, I opened my eyes and realized my fellow devs felt just as accountable as I did (if not more so), and never attempted to pass blame to me. I started asking myself who is really more at fault here; the tester who didn’t test the scenario or the developer who didn’t code to handle it?
I think the tester is 75% responsible for the bug and the developer, 25%. However, the dev probably gets the brunt of the blame because they are a more prominent part of the development team. I would guess more end users have heard of people called software developers than have heard of people called software testers.
These Are Not The Droids You’re Looking For
2 comments Published by Eric Jacobson on May 15, 2008 at 12:13 PMDevs have a special ability to talk testers out of bugs by claiming said bugs cannot be fixed. It goes something like this…
Tester: This UI control doesn’t do what I expect.
Dev: It’s not a bug. It’s a technology constraint. It is not possible to make it do what you expect.
My current AUT uses a third party .Net grid control called “ComponentOne” and I often hear the following:
Dev: It’s not OUR bug. It’s a ComponentOne bug.
I’m fine with these dev responses and even grateful for the info. Where it becomes a problem is when I get soft and don’t log the bug. Criteria for a bug should not include whether we believe it can be fixed or not. Log it! There is so much value in even non-fixable bugs!
Our bug list should capture what is known about the quality of the AUT. If we know the UI widget does not work the way we want it to, let’s document that. Who knows, if these non-fixable UI widget bugs keep getting logged, maybe we can justify using a different UI widget. But a more likely result, from my experience, is that devs will do everything in their power to fix everything on that bug list. And they will suddenly become more creative and write a hack for the technology constraint that earlier looked impossible to overcome.
So don’t be fooled like the storm troopers in front of the Mos Eisley Cantina. If you start hearing “These are not the bugs you’re looking for”, snap out of it!
When I first began getting paid to test software, I was a little confused. I knew it was invigorating to catch scary bugs prior to production but I wasn't really sure how valuable my job was to my dev team or the rest of the world! In fact, I didn't really know if testing software was anything to make a career out of in the first place.
A few years ago I came across Harry Robinson's Bumper Stickers for Testers post on StickyMinds.com. It was at that point that I decided my job as a software tester was valuable (and even a little cool). Harry and all the other software testers who contributed the excellent material on said post inspired me to take pride in my job and now I even sport a couple bumper stickers to show it (see below). If you test software, I encourage you to do the same.
Allen, one of the developers on my team, recently told me about some poorly written bugs he had received from a tester. Allen said,
"Basically these bugs describe the application behavior as a result of user actions working with an application, without any explicit identification of the buggy behavior. This tester wrote many, many bugs with one or two sentences. His bug description pattern was like this:
When I do A, it does B.
Examples:
- When I try to move the video object, the background image stays still. (OK. Where is the bug?)
- When I set the video file to a path located in another course, the file is copied to the current course when rendering the page in IE browser. (To some, this may be the correct behavior. So where is the bug?)
This is an ambiguous way to describe a bug. It frustrates me! "
About a year ago, Allen offered similar candor on several of my bugs. Since then, I have made it a point to force the following two statements into every bug I log.
Expected Results:
Actual Results:
No, it’s not an original idea. But it is an excellent idea that makes it nearly impossible for one to ever log another ambiguous bug. If the tester had used it for one of Allen’s examples above, we might see the following.
Expected Results: The background image moves with the video object.
Actual Results: The background image does not move with the video object. Instead, the background image stays still.
Expected Results/Actual Results. Don't log a bug without these!
RSS
