FeatureA will be ready to test soon.  You may want to think about how you will test FeatureA.  Let’s call this activity “Test Planning”.  In Test Planning, you are not actually interacting with the product-under-test.  You are thinking about how you might do it.  Your Test Planning might include, but is not limited to, the following:

• Make a list of test ideas you can think of.  A Test Idea is the smallest amount of information that can capture the essence of a test.
• Grok FeatureA:  Analyze the requirements document.  Talk to available people.
• Interact with the product-under-test before it includes FeatureA.
• Prepare the test environment data and configurations you will use to test.
• Note any specific test data you will use.
• Determine what testing you will need help with (e.g., testing someone else should do).
• Determine what not to test.
• Share your test plan with anyone who might care.  At least share the test ideas (first bullet) with the product programmers while they code.
• If using automation, design the check(s).  Stub them out.

All the above are Test Planning activities.  About four of the above resulted in something you wrote down.  If you wrote them in one place, you have an artifact.  The artifact can be thought of as a Test Plan.  As you begin testing (interacting with the product-under-test), I think you can use the Test Plan one of two ways:

1. Morph it into “Test Notes” (or “Test Results”).
2. Refer to it then throw it away.

Either way, we don’t need the Test Plan after the testing.  Just like we don’t need those other above Test Planning activities after the testing.  Plans are more useful before the thing they plan.

Execution is more valuable than a plan.  A goal of a skilled tester is to report on what was learned during testing.  The Test Notes are an excellent way to do this.  Attach the Test Notes to your User Story.  Test Planning is throwaway.

This efficiency didn’t occur to me until recently.  I was doing an exploratory test session and documenting my tests via Rapid Reporter.  My normal process had always been to document the test I was about to execute…

TEST: Edit element with unlinked parent

…execute the test.  Then write “PASS” or “FAIL” after it like this…

TEST: Edit element with unlinked parent – PASS

But it occurred to me that if a test appears to fail, I tag said failure as a “Bug”, “Issue”, “Question”, or “Next Time”.  As long as I do that consistently, there is no need to add “PASS” or “FAIL” to the documented tests.  While debriefing about my tests post session, the assumption will be that the test passed unless indicated otherwise.

Even though it felt like going to work without pants, after a few more sessions, it turns out, not resolving to “PASS” or “FAIL” reduces administrative time and causes no ambiguity during test reviews.  Cool!

Wait. It gets better.

On further analysis, resolving all my tests to “PASS” or “FAIL” may have prevented me from actual testing.  It was influencing me to frame everything as a check.  Real testing does not have to result in “PASS” or “FAIL”.  If I didn’t know what was supposed to happen after editing an element with an unlinked parent (as in the above example), well then it didn’t really “PASS” or “FAIL”, right?  However, I may have learned something important nevertheless, which made the test worth doing…I’m rambling.

The bottom line is, maybe you don’t need to indicate “PASS” or “FAIL”.  Try it.

What if the test documentation is written per the detail level in this post… 
Document enough detail such that you can explain the testing you did, in-person, up to a year later.
…and the tester who wrote it left the company? 
Problem #1:
How are new testers going to use the old test documentation for new testing?  They aren’t, and that’s a good thing.

Add test detail as late as possible because things change.  If you agree with that heuristic, you can probably see how a tester leaving the company would not cause problems for new testers.

• If test documentation was written some time ago, and our system-under-test (SUT) changed, that documentation might be wrong.
• Let’s suppose things didn’t change.  In that case, it doesn’t matter if the tester left because we don’t have to test anything.
• How about the SUT didn’t change but an interfacing system did.  In this case, we may feel comfortable using the old test documentation (in a regression test capacity).  In other words, let’s talk about when the write-tests-as-late-as-possible heuristic is wrong and the test documentation author left the company.  If you agree that a test is something that happens in one’s brain and not a document, wouldn’t we be better off asking our testers to learn the SUT instead of copying someone else’s detailed steps?  Documentation, at this level of detail, might be a helpful training aid, but it will not allow an unskilled tester to turn off their brain.  Get it?

Problem #2:
How are people going to understand old test documentation if the tester who left, must be available to interpret?  They won’t understand it in full.  They will probably understand parts of it.  An organization with high tester turnover and lots of audit-type inquiries into past testing, may need more than this level of detail.
But consider this: Planning all activities around the risk that an employee might leave, is expensive.  A major trade-off of writing detailed test documentation is lower quality.

Well, that depends on what your clients need.  How do you know what your clients need?  I can think of two ways:

1. Ask them.  Be careful. Per my experience, clients inflate their test documentation needs when asked.  Maybe they’re afraid they’ll insult the tester if they don’t ask for lots of test documentation.  Maybe they’re intent to review test cases is stronger than their follow-through.
2. Observe them.  Do they ever ask for test case reviews?  If you are transparent with your test documentation, do your clients ever give you feedback?  Mine don’t (unless I initiate it).

Here is what I ask of my testers:

Document enough detail such that you can explain the testing you did, in-person, up to a year later.

Before I came up with the above, I started with this:  The tester should be present to explain their testing.  Otherwise, we risk incorrect information. 

If the tester will be present, why would we sacrifice test time to write details that can easily be explained by the tester?  In my case, the documentation serves to remind the tester.   When we review it with programmers, BA’s, users, other testers, auditors, or if I review it myself, the tester should always be present to interpret.

What if the tester leaves?

I’ll talk about that in the next post.