The “Everlasting Gobstopper”
It strikes me odd that some people on the outside of test automation don’t understand why–or even how–test automation is never really done. And it’s easy to scoff and think that those people are out-of-touch or just don’t get it.
But those people are also thinking: what’s the ROI of something that doesn’t appear to ever end? If it’s never done, are we really getting value out of it?
It’s a good concern to have. If something you’re doing just costs you money, with no benefit, then stop doing it. Spend it on something that provides value.
But in today’s post, I’m going to make the case that even if something is never done, it can still provide tremendous value.
There are 3 main reasons why automation is never done:
- Moving targets
- Huge test suites
- Difficulty in writing it
Aiming at Gooey Stuff
The thing about software is, it’s an organic critter. We’re constantly changing it to account for new technology, new customer requirements, better performance, cleaner code or even bug fixes.
It’s also really complex. Even the simplest looking programs show you the tip of the iceberg. There’s a lot of technical stuff going on beneath the surface–stuff that has to be verified before you, the user, can get your hands on it.
If you picture it like an amorphous gob of goo, where the outer edges are changing but the center is stable, that’s how software behaves sometimes. there’s basic gotta-have functionality, and there’s new stuff on the edges.
Manually testing all of this would be a very expensive task. We need automation to tell us whether the mostly unchanging part–the “stable center”–still works right, so that we can focus on providing the most value.
But we also want to make sure we’re not writing so much that it becomes a huge chore to maintain.
Too Much of a Good Thing?
There’s such a thing as too much automation. Although we could automate every test, we’d end up with a huge maintenance suck.
Eventually, all the time would be spent maintaining the tests, without removing or adding any, and the automation would definitely lose value.
Automation’s goal is not to be served, its job is to serve.
For us to be able to keep up with the changes to the code base, we have to be able to have just enough automation to let us know if something’s wrong in the system, and to let us know quickly.
We also need to be able to produce it quickly.
Trivializing the Solution
I’ve seen, used, and even written a variety of automation frameworks. The ones that work the best are the ones that allow us to focus on the right thing.
There’s an interesting thing that happens mentally to people when automation is hard to write. Test suites get bulkier because people don’t know what’s already in it, and people get “attached” to code they’ve written.
The goal, then, is to make is as easy as possible to create new automation. This both enables us to gain coverage to catch up to the code, and also to delete unnecessary tests.
Trickling vs. Hemorraghing
So let’s say everything’s ideal: we have a small suite of targeted tests, and we can add or remove them at will. It’s also not a lot of time to maintain since there aren’t as many.
Is the automation done, even in this situation? No. Is that bad though? No!
Think about how much time you still save, letting your tests be scheduled to run, or even set up to run when new code is checked in. That’s something to not have to think about.
Think about how, when humans do the same thing over and over again, they get tunnel vision, and stop noticing crucial issues. Automation frees up the mind to get fresh eyes on what’s being tested, and explore the boundaries of the software.
Think about how the time spent on creating or maintaining tests can be focused on code that helps generate more revenue.
Even though automation’s never “done-done”, it’s providing a tremendous amount of value.
If this question comes up at your company, it’s not about whether you need automation or not. The solution is to turn your automation into autom-awesome.
Are you struggling with your test automation? I’ve got some tricks that can help. Contact me below, and let’s talk.