It’s true.  Our job rocks.  Huff Post called it the 2nd happiest job in America this year.  Second only to being a DBA…yawn.  Two years ago, Forbes said testing was #1. 

But why?  Neither article goes in depth.  Maybe it’s because all news is good news, for a tester:

• The System Under Test (SUT) is crashing in QA, it doesn’t work, it’s a steaming pile of…YES!  My testing was valuable!  My life has meaning!  My testing just saved users from this nightmare!
• The SUT is finally working.  Awesome!  It’s so nice being part of a development team that can deliver quality software.  I can finally stop testing it and move on.  Our users are going to love this.

See?  Either way it’s good news.

Or maybe I just spin it that way to love my job more.  So be it.  If you think your testing job is stressful, you may want to make a few adjustments in how you work.  Read my You’re a Tester, Relax post.

During a recent exchange about the value of automated checks, someone rhetorically asked:

“Is automation about finding lots of bugs or triggering investigation?”

Well…the later, right?

• When an automated check passes consistently for months then suddenly fails, it’s an indication the system-under-test (SUT) probably unexpectedly changed.  Investigate!  The SUT change may not be directly related to the check but who cares, you can still pat the check on the back and say, “thank you automated check, for warning me about the SUT change”.
• When you design/code an automated check, you are learning how to interact with your SUT and investigating it.  If there are bugs uncovered during the automated check design/coding, you report them now and assume the automated checks should happily PASS for the rest of their existence.
• If someone is organized enough to tell you the SUT is about to change, you should test the change and assess the impact on your automated checks and make necessary updates.  Doing so requires investigating said SUT changes.

In conclusion, one can argue, even the lamest of automated checks can still provide value.  Then again, one can argue most anything.

Perhaps Jimmy John’s should have hired some software testers before slapping their “Order Online” logo all over the place.

Yesterday, while entering a group order online, I had a little trouble typing my Delivery Instructions in the “memo-size” text box.

The only way to add a drink to your sandwich order was to add a second sandwich.  Um, I only want one sandwich.

I selected the earliest delivery time available:

However, after painstakingly collecting the orders of about 17 people, when I submitted my group order, Jimmy John’s showed me this user validation message:

And prior to me clicking the OK button, Jimmy John’s cancelled my entire order, including sending this email to all 17 people :

There is some additional context here that is too complex for this post.  Suffice it to say, I was irritated that my order was cancelled with no warning.

I called the Jimmy John’s phone number provided on the top of my screen and the dude said, “we have no idea how to retrieve your order, we just get a printout when it’s submitted”. 

In the end, the good people at my Jimmy John’s franchise accepted a fax with screen captures of my original group order (we tried email, but they couldn’t retrieve it) and they delivered the order flawlessly.

Is there a name for this?  If not, I’m going to call it a “fire drill test”.

• A fire drill test would typically not be automated because it will probably only be used once.
• A fire drill test informs product design so it may be worth executing early.
• A fire drill test might be a good test candidate to delegated to a project team programmer.

Fire drill test examples:

• Our product ingests files from an ftp site daily.  What if the files are not available for three days?  Can our product catch up gracefully?
• Our product outputs a file to a shared directory.  What if someone removes write permission to the shared directory for our product?
• Our product uses a nightly job to process data?  If the nightly job fails due to off-hour server maintenance, how will we know?  How will we recover?
• Our product displays data from an external web service.   What happens if the web service is down?

Too often, us testers have so much functional testing to do, we overlook the non-functional testing or save it for the end.  If we give these non-functional tests a catchy name like “Fire Drill Test”, maybe it will help us remember them during test brainstorming.

I Attended John Stevenson’s great talk and workshop at Monday night’s Software Testing Club Atlanta.  I’m happy to report the meeting had about 15 in-person attendees and zero virtual attendees.  Maybe someone read my post.

John is a thoughtful and passionate tester.  He managed to hold our attention for 3 hours!  Here are the highlights from my notes:

• The human brain can store 3TBs of information; This is only 1 millionth of the new information released on the internet every day.
• Over stimulation leads to mental illness.
• John showed us a picture and asked what we saw.  We saw a tree, flowers, the sun, etc.  Then John told us the picture was randomly generated. The point?  People see patterns even when they don’t exist.  Presumably to make sense out of information overload.
• Don’t tell your testing stories with numbers.  “A statistician drowned while crossing a river with an average depth of 3 feet”; Isn’t that like, “99 percent of my tests passed”?
• Don’t be a tester that waits until testing “is done” to communicate the results.  Communicate the test results you collected today?  I love this and plan to blog about it.
• Testers, stop following the same routines.  Try doing something different.  You might end up discovering new information.
• Testers, stop hiding what you do.  Get better at transparency and explaining your testing.  Put your tests on a public wiki.
• Critical thinking takes practice.  It is a skill.
• “The Pause”. Huh?  Really?  So?  Great critical thinking model explained in brief here.
• A model for skepticism.  FiLCHeRS.
• If you challenge someone’s view, be aware of respecting it.
• Ways to deal with information overload:
• Slow down.
• Don’t over commit.
• Don’t fear mistakes.  But do learn from them.  This is how children learn.  Play.
• (Testing specific)  Make your testing commitments short so you can throw them away without losing much.  Don’t write some elaborate test that takes a week to write because it just might turn out to be the wrong test.
• You spend a 3rd of your life at work.  Figure out how to enjoy work.
• John led us through a series of group activities including the following:
• Playing Disruptus to practice creative thinking.  (i.e., playing Scamper.)
• Playing Story War to practice bug advocacy.
• Determining if the 5 test phases (Documentation, Planning, Execution, Analysis, Reporting) each use Creative Thinking or Critical thinking.
• Books John referenced that I would like to read:
• The Signal and the Noise – Nate Silver
• Thinking Fast and Slow – Daniel Kahneman
• You are Not So Smart – David McRaney

At this week’s metric themed Atlanta Scrum User’s Group meetup, I asked the audience if they knew of any metrics (that could not be gamed) that could trigger rewards for development teams.  The reaction was as if I had just praised Planned Parenthood at a Pro-life rally…everyone talking over each other to convince me I was wrong to even ask.

The facilitator later rewarded me with a door prize for the most controversial question.  What?

Maybe my development team and I are on a different planet than the Agile-istas I encountered last night.  Because we are currently doing what I proposed, and it doesn’t appear to be causing any harm.

Currently, if 135 story points are delivered in the prior month AND no showstopper production bugs were discovered, everyone on the team gets a free half-day-off to use as they see fit.  We’ve achieved it twice in the past year.  The most enthusiastic part of each retrospective is to observe the prior months metrics and determine if we reached our “stretch goal”.  It’s…fun.  Let me repeat that.  It’s actually fun to reward yourself for extraordinary work.

Last night’s question was part of a quest I’ve been on to find a better reward trigger.  Throughput and Quality is what we were aiming for.  And I think we’ve gotten close.  I would like to find a better metric than Velocity, however, because story point estimation is fuzzy.  If I could easily measure “customer delight”, I would.

At the meeting, I learned about the Class of Service metric.  And I’m mulling over the idea of suggesting a “Dev Forward” % stretch goal for a given time period.

But what is this nerve I keep touching about rewards for good work?

On weekends, when I perform an extraordinary task around the house like getting up on the roof to repair a leak, fixing an electrical issue, constructing built-in furniture to solve a space problem, finishing a particularly large batch of “Thank You” cards, or whatever…I like to reward myself with a beer, buying a new power tool, relaxing in front of the TV, taking a long hot shower, etc.

Rewards rock.  What’s wrong with treating ourselves at work too?

Warning: This has very little to do with testing.

Additional Warning: I’m about to gripe.

I attended the 3rd Software Testing Club Atlanta meetup Wednesday.  Some of the meeting was spent fiddling with a virtual task board, attempting to accommodate the local people who dialed in to the meeting.

IT is currently crazy about low tech dashboards (e.g., sticky notes on a wall).  But we keep trying to virtualize them.  IMO, virtualizing stickies on a wall is silly.  The purpose is to huddle around, in-person, and ditch the complicated software that so often wastes more time than it saves.

IMO, the whole purpose of a local testing club that meets over beer and pizza is to meet over beer and pizza...in person, and engage in the kind of efficient  discussion that is best done in person.  Anything else defeats the purpose of a “local” testing club.  If I wanted to dial in and talk about testing over the phone, it wouldn’t have to be with local people. 

I’m sad to see in-person meetings increasingly get replaced by this.  But IMO, joining virtual people to real-life meetings, can be even worse.  Either make everyone virtual or make everyone meet physically.

Yes, I’m a virtual meeting curmudgeon.  I accept that virtual connections have their advantages and I allow my team to work from home as frequently as three days a week on a regular basis.  But I still firmly believe, you can’t beat good old fashioned, real-life, in-person discussions.