Okay, we (tester) found a bug and figured out the repro steps. Can we log it now or should we investigate it further? Maybe there is an error logged in the client’s log file. Maybe we should also check the server error logs. And wouldn’t the bug description be better if we actually figured out which services were called just before said error? We could even grab a developer and ask them to run our test in debug mode. Or better yet, we could look at the code ourselves because we’re smart tester dudes, aren’t we?

If you thought I was going to suggest testers do all this extra stuff, you’re wrong. I've read that we should. But I disagree. We’re the tester. We test stuff and tell people when it doesn’t work. We don’t have to figure out why. If we’ve got the repro steps, it may be okay to stop the investigation right now. That’s the whole point of the repro steps! So the Dev can repro the bug and figure it out.

Look, I get it...we’re cool if we can dig deep into the inner workings of our AUT and maybe we’re providing some value added service to our devs. The problem is, we’re not the devs. We didn’t write it. Thus, we are not as efficient as the devs when it comes to deep investigation. And time spent investigating is time NOT spent on other testing. For me, everything must be weighed against the huge number of tests I still want to run.

So unless you’ve run out of good tests, don’t spend your time doing what someone else can probably do better.

What do you think?

3 comments:

  1. P.J. Cornelius IV said...

    I concur that the majority of the time if you're positive and have a set of steps that is reproducible you can stop right then and there. No need to waste any more time.

    However you mentioned whether you should look in the error logs or not. As long as the developer can reproduce it themselves given the repro steps it's typically not a problem since we can go and look at the log file ourselves.

    However at times bugs which might seem to be consistently reproducible is really only reproducible by you due to the data setup, specific client/server configuration, the phase of the moon, wind chill factor, current number of Lindsay Lohan's boyfriends, etc. In which case then information in the client/server logs (especially the stack trace which tells us which line of code caused the error and the exception) is paramount to pinpointing a bug that is *not* reproducible in the dev. environment and only appears in another environment.

    And when the bug is logged sometimes it might take a while to go through the loop from team lead, to developer, then on to another developer, etc. By that time the logs could be gone by then if we can't reproduce it in the dev. environment. And if not gone then the logs are so huge that without an exact time it might be difficult to go back to find the real error. So the bug then ends up typically being rejected as non reproducible.

    So it's a tradeoff and I think if you're unsure and have second thoughts as to the whether the info you're logging is enough then just let the developer know and he/she will inform you if they need more info. Although some developers would rather that you just log the bug and not bother them, me personally I don't mind since I feel that testers are helping me make the code more robust and I'd rather take a look at something then that can be reproduced rather than try to figure out what you were doing at the time. I guess you'd need to know the developer's personalities and how they like to work since it can be a touchy issue.

    Hmm just noticed this comment is probably longer than your blog post. However, I felt a need to post a comment ... :)

  2. Alexey Bulat said...

    Sorry, but I don't agree with your point.
    Yes we are testers and we should test and we should do our best in this way.

    At the first, we can think from position of rates. Who does usually get higher salary rate devs or testers? I think devs, so from this point of view, if tester doesn't fully investigate the reason of the error developer will spend more time for fixing. So the price of bug will be higher.

    The second, all issues has their requiremns. Bug report is not an exception. As minimum we should describe: steps, result, expected results, user environment (browser name and version for web testing, OS name and version), version of application. If tester has described all these it will be enough even without client or server log with errors.

  3. Eric Jacobson said...

    Alexey,

    You make an interesting point about salaries. Yes, if the dev costs significantly more than the tester than it does seem more practical for the tester to spend more time on investigation. Lots of variables to consider...

    I agree with your last paragraph.

    BTW - Sorry I didn't post your comment sooner, I just back from an Ireland vacation and I was too lazy to check this blog.



Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.
No part of the content or the blog may be reproduced without prior written permission.