Does your test manager ever test? They should.

I recently got promoted to test manager. About three months in, I started to get used to delegating much of the testing tasks. I have to admit, it was nice to focus on the big picture for a while. I began sounding like a manager, being more interested in status rather than test value.

When one of my testers took a vacation to India and the other two got sick, I had to jump in where my testers left off and complete a variety of testing activities. It was like getting smacked in the face.

• In some places where I thought the tester was dragging, I discovered legitimate test impediments.
• In some places where I believed the tester excuses, I found tester misunderstandings or poorly designed tests.
• In all areas, I experienced the stress of uncertainty, the constant decision making, and the thrill of finding important problems.
  

The best way to really grok a job is to perform it yourself. Experiencing the act of testing is different than observing it or hearing summaries of it. So...

• Testers, the next time you take a personal day off work, ask your manager to be your backup and actually do some of your work while you’re out.
• Managers, offer to jump in and be the backup tester.

I caught two Stareast James Bach talks. The “The Myths of Rigor” dealt with when to use rigor and when not to. The main idea (as I understood) was to use more rigor upstream and less downstream. For example, if you're coaching a new tester, you may want to provide them with checklists and lots of details, then encourage them to begin thinking without following said checklists once they grok the concept. Experts are bad at explaining what they know and learners tend to say they understand when they don't; the checklists and details may help, but only at the beginning.

This clicked for me when James asked, “Have you ever written a process document and then not followed it?”. Absolutely! I'm smart enough to understand when to break the rules. How about a test case? Of course! Per James...

• Rigor at the outcome of a test is optimized for a static well-known world.
• Rigor at the planning of a test helps you adopt to a changing world.
  

Writing test cases is valuable as long as we don’t become victims of what James calls “Pathetic Compliance”; following the rules just so we don’t get yelled at, even though we don’t understand the rules. The value in writing test cases is:

• they are excellent for a quick review before test sessions to get your head straight
• they are a good tool for discussing tests and understanding each other
• creating them helps learning
  

So write test cases but don’t force yourself to use them.

BTW - James Bach is working on a book about how to coach software testers.


The second of James Bach’s talks was a keynote, “The Buccaneer Tester: Winning Your Reputation”. This seemingly dull topic is actually important. The main takeaway for me was:

Making yourself unremarkable does not keep your job safe.

Per James, being good at testing and getting credit for your work are both optional. Sadly, I’ve worked with lots of unremarkable testers. His advice, if you choose to become remarkable:

• Determine what mix of tester skills you have that nobody else has.
• Use the above to come up with some kind of vision about testing. It doesn’t even need to be a good vision; bad visions can also give you a reputation.
• Take a stand on an issue.
• Participate in public (volunteer) testing.
• Write, teach, speak, study, and experience more types of testing
  

After a long day at one of the best software testing conferences I’ve attended, I opened the door to my hotel room and found it full of Stareast keynote speakers, track presenters, and five author/editors of testing books I had recently been reading. It was like some creepy tester fantasy. About half of my favorite tester thinkers had gathered into my hotel room and were conducting lightning talks in front of a flip chart and flat screen TV, some 15 feet from the Queen-sized Murphy bed I sleep in.

This is one of the reasons I love being a software tester. After a bit of networking during my first day at Stareast, I found myself invited to dinner with the Stareast Rebel Alliance, a group of testers who are becoming active in the speaker circuit, blogosphere, and Twitter, attempting to improve the craft of testing. They answered testing questions, gave me an Alliance t-shirt, tried to buy me dinner, and made me feel like family. Special thanks to Alex Kell for introducing me to this crowd.

When Matthew Heusser mentioned he needed a place to host a tester gathering the following night, I suggested my hotel room. After all, the Rosen Shingle Creek had overbooked and given me the parlor suite (maximum occupancy 78). True to their plan, this ambitious group of testers met after the conference and gave lightning talks, provided support and candor, challenged each other with testing games, ate, drank, and were merry. Jon Bach and Michael Bolton were each at different tables, using dice games and puzzles to teach testers better thinking. Adam Goucher, Lanette Creamer, and Matthew Heusser practiced newish lightning talks on tester roles. Shmuel Gershon demonstrated a new session based testing tool he is writing (to be less disruptive to the tester’s concentration). Justin Hunter gracefully demoed his Hexawise test case generation tool. Lisa Crispin and Janet Gregory, co-authors of Agile Testing were there asking questions and providing support. Tim Riley happily discussed various Mozilla testing processes. Agile testing experts Dawn Cannan and Elizabeth Hendrickson also showed up to defend and explain their ideas.

There were many other testers who came and went that night, all were polite, interesting, modest, and fun to hang out with. The last of them left around 2:30 AM some time after winding down and watching a few choice TED talks. I went to sleep with the thick smell of carry-out Indian Food next to my bed.

The next day I woke up and walked barefoot across my 78 occupancy room. I stepped on something. About 5 hours earlier, Michael Bolton was shoving Smartfood popcorn into his mouth and spilling it on the floor. He picked some up saying, “I wouldn’t want you to get Smartfood Foot tomorrow”. I guess he missed a piece.

Lanette Creamer has an excellent overview of the conference on her testyredhead blog. I'll list my personal take-aways in future posts.

Our most complex AUT has no shortage of production bugs. They’re discovered almost daily and our support team forwards them to the rest of the team. These bugs get reported with little factual detail and it’s up to a BA, Dev, or Tester to figure out the repro steps.

Our informal process is, the first person to reply saying “I’m on it” owns the issue and gets to be the hero who figures it out. Determining the elusive repro steps combines many skills; interviewing oracles, listening to users, acting like users, pulling audit trails from the DB using SQL, examining artifacts like user screen captures and error log files, and tracking down user stories or requirements.

Last week one of my testers stopped by my cube, grinning ear-to-ear, and said “I’m so excited! I just figured out the repro steps!” (to a really really challenging prod bug). She sent her repro steps out to the team, they were correct, and within minutes she was thanked and declared a rock star by various team members.

Determining the exact minimum repro steps, is there anything more exciting for a tester?

…well, hopefully. But cracking repro steps is pretty darn exciting!

The above question was asked in response to my Do Developers Make Good Testers? post. Since I am in the process of hiring another tester I thought I would take a stab at it. These are qualities for a fairly generic software testing position.

A good software tester…

• Constantly asks, “What is the best test I can execute right now”.
• Can log unambiguous bugs with clear repro steps that make the main problem obvious with few words.
  
• Is not distracted by their understanding of developer decisions. Just because the tester may understand certain technology constraints motivating dev solutions, the tester’s mission is never to defend the AUT (see my post, What We Can Learn From Dumb Testers). It is to communicate how the AUT currently works, in areas that matter right now.
• Has the capacity to understand the stakeholders’ business.
• Is technical enough to see how one component of a system affects the entire system.
• Has keen problem solving skills. They can control multiple variables until locating the problematic variable. They have just enough persistence without having too much. They know when to quit and move on.
• Is an expert communicator and listener who demands complete understanding.
• Is humble enough to ask all questions (even stupid ones) but cynical enough to seek answers from multiple sources (trust but verify).
• Is organized enough to follow through with tasks, while at the same time noting potential future tasks.
• Is capable of isolating observed software behavior, within an ocean of dependencies and communicating those behaviors to the team. They can look at components of an incomplete system and determine actual pros and cons by imagining the complete system.
• Respects fellow developers and BAs. Understands the harder the tester works, the better the developers/BAs look.
  
• Is enthusiastic when finding pre-production bugs but depressed when users find post-production bugs.
• Can handle stressful deadlines, make quick decisions, and give up preferred processes for those that ultimately are in the stakeholders' best interest.
• Is an active participant in the software tester community, reads testing books/blogs, and participates in local test groups.
• Has good work ethic; can meet deadlines or communicate they will be missed, works more than 40 hour weeks when necessary, is organized and professional, cares about the team’s success, honest, follows mandatory work procedures, is SOX compliant, etc.
  

What have I missed?

I have an open headcount for the highest level tester position at my company. We are hoping to get someone with test automation skills. Most of the candidates are making me yawn. Few are enthusiastic enough to have read anything about their own craft, and fewer are interested in deep discussions.

One resume that grabbed my interest was from a career developer that claims to want to try her hand at being a tester. She is mainly interested in writing test automation code...but she never has. Prior to an interview, my devs and I were enthusiastic. Afterwards, we realized this candidate had no testing experience (other than experimenting with unit tests) and most feared our team could end up with a good developer but a weak test automation stack instead of a good tester who could provide instant gratification via sapient UI tests.

A dev wanting to cross over... How rare of a thing is this? Am I missing a great opportunity by not hiring the dev? Or is this a dev that just couldn’t cut it as an application developer…hoping to cut it as a test automation developer?

I can’t help but wonder, is it easier to teach a developer how to be a good test automator or to teach a good manual tester how to be a good test automator?

A bunch of us participated in three days of “Agile Boot Camp”. Speaker/Agile Coach/President of DavisBase, Steve Davis came and attempted to enlighten our department by teaching us what Agile development is all about. My department has been practicing its own flavor of Agile development for about four years but many of us have felt it’s time to adopt more Agile ideals. For starters, QA is still one iteration behind dev.

For me, the class was mostly info I had heard before. I hoped to collaborate with others and help them grasp new ideas. I was frustrated about 50% of the time as I watched many of my higher level team members look up from their Blackberrys and iPhones throughout the class to say, “Right, Steve, but that’s not the way we do things here”. Sigh.

Nevertheless. Here are some of the tidbits and ideas I wrote on the back of my tent card. Many of these are related to my new role as QA Manager.

• Make my testers feel like they are part of the solution. Give them missions and get out of the way. Don’t assign specific tester tasks. Let them work for their team. I will say “I did not hire you to please me. I hired you to please your team”.
• If the above is true, what is my role as a QA Manager? To mentor, teach, and figure out how to make my test team the best team there is.
• Assign each of my testers to a permanent dev team instead of bouncing them between teams. Locate each tester with their dev team. The longer they are a team, the more likely they will develop a cadence (natural rhythm). The cadence will allow them to stop worrying about process and concentrate on what they do best…testing!
• Get the hardest tests done first.
• Write the high level tests during the Feature walkthrough.
• Writing user stories. Don’t get too far ahead on capturing detail. If you do, you are approaching the waterfall methodology.
• If you are not going to have enough time to test, bring that up in your daily scrum ASAP so others can help.
• Daily scrum. Stop looking at the lead or scrum master as you report. Look at your team members. Your report is for them.
• Don’t wait until the end of the iteration to get anxious. Rally around the goal from day 2. Keep the energy high.
• Eric Idea: Post test status inside bathroom.
• Eric Idea: Testers evaluate each other. (this may not work if they don’t work together.)
  

I’ll report back later in the year with updates on how we’re doing.