While helping some testers, new to automation, I found myself in the unexpected position of trying to sell them on the idea that all test methods should be mutually exclusive. Meaning, no automated check should depend on any other automated check…automated checks can run in any order…you can run them all, in any order, or you can run just one.
If I could take one test automation rule to my grave, this would be it. I had forgotten that it was optional.
I know, I know, it’s seems so tempting to break this rule at first; TestA puts the product-under-test in the perfect state for TestB. Please don’t fall into this trap.
Here are some reasons (I can think of) to keep your tests mutually exclusive:
- The Domino Effect – If TestB depends on TestA, and TestA fails, there is a good change TestB will fail, but not because the functionality TestB is checking fails. And so on.
- Making a Check Mix – Once you have a good number of automated checks, you’ll want the freedom to break them into various suites. You may want a smoke test suite, a regression test suite, a root check for a performance test, or other test missions that require only a handful of checks...dependencies will not allow this.
- Authoring – While coding an automated check (a new check or updating a check), you will want to execute that check over and over, without having to execute the whole suite.
- Easily Readable – When you review your automation coverage with your development team or stakeholders, you’ll want readable test methods. That usually means each test method’s setup is clear. Everything needed to understand that test method is contained within the scope of the test method.