09 Aug Automation Will Never Replace Exploration

We here at Clear Sky are obviously kind of huge fans of automation, be it test automation or otherwise.

Having lived through much proverbial bloodshed and having witnessed too many large-scale applications drown in their own regression bugs, we feel our obsession is well merited. Most organizations we’ve worked with had a large chunk of the allotted development time reserved for regression testing alone. We are convinced that time could have been better used to provide customer value. Automation helps reduce that waste, and though you don’t get results overnight, you’ll likely see millions of dollars of savings in the first year alone.

And that success, advertised up the flagpole, usually causes that pendulum to swing so fast, insurance rates skyrocket because of too many whiplash cases. (Clearly, I’m exaggerating for effect but it sure feels that way.) That fervent resistance to any automation gave way to replacing 100% of manual testing with that same automation. But that’s just as bad as no automation!

Like with most things in life, there must be a balance. There should always be some people and time dedicated to conducting exploratory testing.  Using the system in the mindset of a production user is not just helpful, it’s necessary. This kind of testing should include different personas that use our system, including the “nice user”, the “grumpy Gus” and the clueless users, who consider a simple link akin to a doctoral thesis. Of course, always incorporate the all-around malicious user who’s out to break into, or just break, the system.

Beyond testing, running the application manually in the mindset of a real user can yield other benefits, like:

Finally, exploratory testing can provide insights on which features should be removed (yes, you read that right). This type of human interaction and exploration, in our humble opinion, just can’t be fully replicated with automation… not yet anyway.