Sometimes, the most feasible way to test something, is to let it soak in an active test environment for several weeks. Examples:
- No repro steps but general product usage causes data corruption. We think we fixed it. Release the fix to an active test environment, let it soak, and periodically check for data corruption.
- A scheduled job runs every hour to perform some updates on our product. We tested the hourly job, now let’s let it run for two weeks in an active test environment. We expect each hourly run to be successful.
Per Google, soak testing involves observing behavior whilst under load for an extended period of time. In my case, load is normally a handful of human testers, as opposed to a large programmatic load of thousands. Nevertheless, the term is finally catching on within my product teams.
Who cares about the term? I like it because it honestly describes the tester effort, which is very little. It does not mislead the team into thinking testers are spending much time investigating something. It’s almost like not testing. But yet, we still plan to observe from time to time and eventually make an assessment of success or failure.
Be sure to over-annunciate the “k” in “soak”. People on my team thought I was saying “soap” test. I’m not sure what a soap test is…but I’m sure it exists too!