I have an open headcount for the highest level tester position at my company. We are hoping to get someone with test automation skills. Most of the candidates are making me yawn. Few are enthusiastic enough to have read anything about their own craft, and fewer are interested in deep discussions.

One resume that grabbed my interest was from a career developer that claims to want to try her hand at being a tester. She is mainly interested in writing test automation code...but she never has. Prior to an interview, my devs and I were enthusiastic. Afterwards, we realized this candidate had no testing experience (other than experimenting with unit tests) and most feared our team could end up with a good developer but a weak test automation stack instead of a good tester who could provide instant gratification via sapient UI tests.

A dev wanting to cross over... How rare of a thing is this? Am I missing a great opportunity by not hiring the dev? Or is this a dev that just couldn’t cut it as an application developer…hoping to cut it as a test automation developer?

I can’t help but wonder, is it easier to teach a developer how to be a good test automator or to teach a good manual tester how to be a good test automator?

A bunch of us participated in three days of “Agile Boot Camp”. Speaker/Agile Coach/President of DavisBase, Steve Davis came and attempted to enlighten our department by teaching us what Agile development is all about. My department has been practicing its own flavor of Agile development for about four years but many of us have felt it’s time to adopt more Agile ideals. For starters, QA is still one iteration behind dev.

For me, the class was mostly info I had heard before. I hoped to collaborate with others and help them grasp new ideas. I was frustrated about 50% of the time as I watched many of my higher level team members look up from their Blackberrys and iPhones throughout the class to say, “Right, Steve, but that’s not the way we do things here”. Sigh.

Nevertheless. Here are some of the tidbits and ideas I wrote on the back of my tent card. Many of these are related to my new role as QA Manager.

• Make my testers feel like they are part of the solution. Give them missions and get out of the way. Don’t assign specific tester tasks. Let them work for their team. I will say “I did not hire you to please me. I hired you to please your team”.
• If the above is true, what is my role as a QA Manager? To mentor, teach, and figure out how to make my test team the best team there is.
• Assign each of my testers to a permanent dev team instead of bouncing them between teams. Locate each tester with their dev team. The longer they are a team, the more likely they will develop a cadence (natural rhythm). The cadence will allow them to stop worrying about process and concentrate on what they do best…testing!
• Get the hardest tests done first.
• Write the high level tests during the Feature walkthrough.
• Writing user stories. Don’t get too far ahead on capturing detail. If you do, you are approaching the waterfall methodology.
• If you are not going to have enough time to test, bring that up in your daily scrum ASAP so others can help.
• Daily scrum. Stop looking at the lead or scrum master as you report. Look at your team members. Your report is for them.
• Don’t wait until the end of the iteration to get anxious. Rally around the goal from day 2. Keep the energy high.
• Eric Idea: Post test status inside bathroom.
• Eric Idea: Testers evaluate each other. (this may not work if they don’t work together.)
  

I’ll report back later in the year with updates on how we’re doing.

Tester Reputation Cheats

How well do you know your testers? There are several tester stunts I have been tempted to pull (and sometimes have). The act of testing is difficult to monitor so it is easy for testers to spoof productivity. If you catch yourself doing these…don’t.

• Whenever team members stop by your desk, make sure you have a GUI automation test running on one of your boxes. It looks really cool and gives the appearance that you are an awesome tester (they’ll never know it’s just a looping record/playback test with no verifications).
• Your manager needs to see test cases. How about just copying the requirements into something called a test case?
• You didn’t stumble upon that bug (yes you did). It came from a carefully planned test.
• There is no evidence of what tests you executed because you kept track of them locally (not really). You will upload the tests to the repository when you have time (yeah, right).
• You didn’t forget to log that bug your dev is asking about (yes you did). You are still reducing the repro steps down to the bare minimum due to some variables that may be related.
• You didn’t forget to execute those tests (yes you did), you choose not to execute them because you performed a risk assessment matrix resulting in a lower priority for said tests.
• When running performance tests using your wrist watch, record time span values out to the millisecond. It appears more accurate.
• Does your manager review your test cases or just see how many you have written? If it’s the later, find the copy/past option in your test case repository and change one value (e.g., now pass in “John”, now pass in “Pat”, now pass in “Jill” etc.). Dude, you just wrote 30 test cases! It looks great on the metrics summary page.
• If you’re lost at the Feature Walkthrough meeting, periodically raise your hand and ask if there are any “ility” requirements (e.g., Scalability, Usability, Compatibility, etc.)…it just sounds cool to ask.
• You don’t have a clue how this feature is supposed to work. If you wait long enough, another tester will take care of it.
  

Have you noticed any additional stunts testers pull?

You probably think this is just another post about how important it is to test everything before it goes to production. Nope.

Testers are too often expected to test things they don’t have a chance in hell at testing. Testing for the sake of process or perception, provides little to no value and degrades the whole trade of testing.

Sometimes when people ask,

“What are the testers going to do to test it?”, I respond,
“Nothing…we’ll just rubber-stamp it.”

Devs usually laugh because they know the bug risk is ridiculously low or the tester does not have the skills to test anything beyond what the dev already tested. However, other testers, managers, or BAs react with horror.

“Rubber-stamp it? Blasphemy! You’re the tester. EVERYTHING must be tested by you before going to production!”

The term “rubber-stamping” invokes negative reactions because it brings up the mental image of the desk clerk stamping document after document without paying any attention to what is on them…like the tester marking “Verified” on the bug or feature they didn’t really do anything to test. But that’s why I like the term! I’m trying to be honest about what value the tester added…none.

Here are some examples where the rubber-stamper-tester is justified:

• The tester has inadequate skills to perform the tests but has interviewed someone else on the team (usually the devloper) who did test.
• The test is not feasible to recreate in a non-prod environment (e.g., a complex multi-user scenario, custom PC environments, unknown repro steps)
• The patch can only be verified in debug mode, using complex timing and coordination requiring breakpoints and other unrealistic tricks. Even devs may skip testing if regression tests pass.
• If critical functionality is broken in prod, we may decide to release a fix without testing it. Speed becomes paramount in this scenario. We are smart...it is possible that we are smart enough to see a logic error, make the fix, take a deep breath and release to prod without the overhead of testing. After all, it can’t get any worse in prod, right?
  

It’s fun to say our job as testers is to be cynical, to not trust anybody. But we shouldn’t abuse that belief and become mere team bottle-necks, either.

Does one require more start-up time than the other?

My department is gearing up for a spike in development this summer. We plan to use temporary, contractor developers and testers.

IMO, the contractor devs do not need to know nearly as much about our systems as the contractor testers do. The devs will be able to make their updates with limited business domain or system knowledge. However, the testers will need to go way beyond understanding single modules; they will need extensive business domain and system knowledge in order to determine what integration tests to write/execute.

Some on my team have a notion that testers can come in and be valuable by running simple tests, with limited knowledge. It scares me so much I would almost consider giving up my entire summer, moving into the office, and doing all the testing myself.

I’m also wondering if contractor testers should be paired with veteran devs and vice versa. If we have contractor testers working with contractor developers, as is the plan, it sure seems like a recipe for disaster.

Have any of you experienced something similar? What advice do you have for me?

I’ve spent a good deal of time underground the last 13 years…literally. One of my favorite weekend activities is caving. New caves are discovered nearly every weekend in the northwest Georgia area and responsible cavers survey these caves, make maps, then submit the data to their state’s speleological survey library.

Cavers are very methodical when it comes to finding virgin passage. Underground ethics specify that cavers survey (with tape, compass, clinometer, and sketchbook) the new passage as they explore it. It is frowned upon to just run through a new cave without performing a proper survey on the way in. Exploring without an initial survey is known as “scooping” or “eye-raping” a cave and it is a sign of an irresponsible caver (sometimes called a “spelunker”).

The responsible caver learns everything about the new cave by surveying as they go. The survey process forces them to examine all features of the passage carefully, often discovering new leads, which otherwise would have been missed. This patient approach keeps the caver fresh with anticipation about the wonderful cave lying ahead.

The irresponsible caver, who runs down virgin passage into the dark unknown, only experiences the obvious way forward. They assume they will backtrack to check for leads. In practice, they may grow fatigued or bored with the cave and never return. They have not collected enough data to qualify the cave with the state survey. They can brag to friends about a deep pit and borehole passage. But they cannot tell other cavers to bring a 250-foot rope because the pit is 295-feet deep, or that the big formation room is a half mile in on the northeast end of a 40-foot wide dome room. They don’t know which leads have been checked or how likely it is that this cave drains into a nearby cave further down the mountain. They have no hard facts about the cave; only memories, which fade very quickly.

A tester's approach to new AUT (Application Under Test) software features should be much the same as those cavers who survey as they explore. As a tester, my tools are my tests. And yes, for complex scenarios, I like to write the test before I perform it. At times, I want to scoop the application, to find the bugs before the other team members do. But I try to reign myself in. I keep track of what I have checked as I go in. I remember how satisfying it is to present team members with a list of tests performed and their results; so much more satisfying than saying “I’m done testing this”.

If your AUT has logic that checks for null values, make sure removed values get set back to null. If they get set to blank, you may have a nice crunchy bug.

Here are the pseudo steps for the test case using a generic order form example. But you can use anything in your AUT that programmatically makes a decision based on whether or not some value exists.

Feature: Orders with missing line item quantities can no longer be submitted.

1. Create a new line item on an order. Do not specify a quantity. Crack open the DB and find the record with the unpopulated quantity. Let’s assume the quantity has a null value.
2. Populate the quantity via the AUT .
3. Clear the quantity via the AUT. Look at the DB again. Does the quantity have a blank or null value?
   
   Expected Results: Quantity has a null value.

If your dev only validates against null quantity values, your user just ordered nothing...yikes!