tag:blogger.com,1999:blog-8951904624959546499.post8706692422659651326..comments2024-02-22T05:36:59.121-05:00Comments on Test This Blog - Eric Jacobson's Software Testing Blog: Deciding What Not To TestEric Jacobsonhttp://www.blogger.com/profile/08216361684596485033noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-8951904624959546499.post-82876210146362732882008-12-08T13:25:00.000-05:002008-12-08T13:25:00.000-05:00Michele,Thanks for sharing! It's sad that we can'...Michele,<BR/><BR/>Thanks for sharing! It's sad that we can't tell such things to managers.<BR/><BR/>You raise an interesting question. high level breadth vs. low-level targetted areas. Hmmmm...Eric Jacobsonhttps://www.blogger.com/profile/08216361684596485033noreply@blogger.comtag:blogger.com,1999:blog-8951904624959546499.post-90174439336622392512008-12-08T13:13:00.000-05:002008-12-08T13:13:00.000-05:00Michael,Your heuristics argue your point well (exc...Michael,<BR/><BR/>Your heuristics argue your point well (except c). I guess thinking about possible tests can take place quicker than I initially thought. I think your “best compared to what?” question is what really made me think, though.<BR/><BR/>Okay, so my original thinking was:<BR/><BR/>A test that results in the most valuable information jumps out for the tester. Execute and repeat the above until some stopping heuristic occurs.<BR/><BR/>The revised approach may be:<BR/><BR/>A test that results in the most valuable information is determined by the tester after comparing it with tests that result in less valuable information. Execute and repeat the above until some stopping heuristic occurs.Eric Jacobsonhttps://www.blogger.com/profile/08216361684596485033noreply@blogger.comtag:blogger.com,1999:blog-8951904624959546499.post-70279480254904992762008-12-05T14:42:00.000-05:002008-12-05T14:42:00.000-05:00I would like to think, as a good tester, I will co...<I>I would like to think, as a good tester, I will come up with the best tests first.</I><BR/><BR/>I'd like to think that too. But best <I>compared to what</I>?<BR/><BR/>Here's a set of heuristics for you to consider; I think they represent some of the essence of Rob's perspective on this (which I share).<BR/><BR/>a) Thought is fast and cheap.<BR/>b) A test can be fast and cheap, but usually not as quick as a thought.<BR/>c) A test reveals information that a thought might not.<BR/>d) We can think not only about tests, but about categories of tests, about categories of test techniques, about oracles, about coverage. So we can consider (and optionally accept or reject or prioritize) a whole slew of tests at once.<BR/>e) We can't ever be sure we're right, but if we pause even for a moment to consider some alternatives, we can (fallibly) reduce the chance that we'll miss something important.<BR/><BR/>---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-8951904624959546499.post-9299550991984021742008-12-05T13:40:00.000-05:002008-12-05T13:40:00.000-05:00"His main point was, it’s better to not test certa..."His main point was, it’s better to not test certain things because you decided not to, rather than because you ran out of time."<BR/><BR/>In my imagination: The manager calls me into his office and asks me why the customers are calling about feature X not working correctly.<BR/><BR/>I tell him, "I decided not to test that because I wanted to focus my time on feature Y."<BR/><BR/>I cannot imagine that being an acceptable answer. <BR/><BR/>It is better to me that I run out of time. It is possible to go for high-level breadth of the application in test and not cover the depth of everything. In my opinion, covering the application is better than selecting what to test or what not to test. Developers miss bugs in what they themselves have designed. And I am supposed to determine, based upon what has been told to me by developers, that only areas A,B, and C have fixes or changes in them? I have found bugs in areas of an application where the developers swear they did not change any code. Sometimes bugs move, testers know that. <BR/><BR/>The thought of selecting not to test certain things because I decided not to makes me picture my manager dressed up like Dirty Harry saying, "Do you feel lucky, punk?" And , like Dirty Harry, my manager is a pretty good guy.<BR/><BR/>Myself, I would rather run end to end in the application at a high level (breadth) than have my focus in some areas and not in others. And that is what I try to do for every project that I am on. I prefer to continue to build my testing skills and use the Rapid Testing approach than to select what I will and won't test, based on what? <BR/><BR/>Interesting topic, Eric. Made me have to process the reason why it was not settling well with me.Michele Smithhttps://www.blogger.com/profile/05639189538178022310noreply@blogger.comtag:blogger.com,1999:blog-8951904624959546499.post-83506744772650791362008-12-05T12:55:00.000-05:002008-12-05T12:55:00.000-05:00Zach,Really? Which part was helpful? When you ge...Zach,<BR/><BR/>Really? Which part was helpful? When you get out of your crunch please share your tips. Thanks!Eric Jacobsonhttps://www.blogger.com/profile/08216361684596485033noreply@blogger.comtag:blogger.com,1999:blog-8951904624959546499.post-74568105490038260122008-12-05T02:33:00.000-05:002008-12-05T02:33:00.000-05:00Man, this was a very timely post. I needed to read...Man, this was a very timely post. I needed to read something to give clarity to test crunch happening right now. Boy. This was a Godsend.Zachary Fisherhttps://www.blogger.com/profile/11364709737097199211noreply@blogger.com