The longer I do it, the more I realize that testing software is a lot like fishing.
Now I’m no expert on fishing, but I do know some things about fish:
- different species of fish like certain kinds of bait
- some fish live only in certain areas
- fish have a lifespan and will only be available at certain times of the year
- particular fish require different equipment to catch them properly
All four of these points need to be considered in order to have a successful catch. I don’t know, maybe there are more than 4. But let’s consider an example: fishing for marlin.
A marlin is a deep-sea fish. Deep-sea means salt water.
They’re also a big honking fish, so it requires a deep-sea fishing rig.
There’s probably a good time to catch them–a time when they’re at peak maturity, or mating, or closer to the surface or something.
Being big fish, they eat other sizable animals. Google says they like tuna.
Huh. I just realized, an acronym can be used for this. BELT, for Bait, Equipment, Location and Timing. That’s pretty cool. We’ll come back to that later.
Anyway: All four conditions have to be met in order to catch this fish. If even one is wrong, the chances are slimmer that there will be a successful catch. Fishing in a freshwater lake, using a wimpy fishing pole, fishing at the wrong time of year or fishing with worms instead of tuna, will zonk the success rate.
In the same sense, when you begin the hunt for a bug, consider the following:
- not all bugs are equal–there are certain kinds, or species, of bugs. What kind do you want to catch?
- not all bugs exist everywhere–bugs of one ilk are easy to find in one place, but rare in others. Where are you looking for bugs?
- some bugs only exist at particular times. Are you looking at the right time?
- some bugs won’t reveal their presence unless you provide the right stimulus. Are you using the right tools and data for the job?
So back to the BELT acronym–Bait, Equipment, Location and Timing. And, time for another example.
Let’s say the species of bug you’re looking for is a crash in a C++ application. Crashes can be triggered when bad or unitialized data is passed between functions, or if the application reads something wonky into itself.
What would be the bait in this case? How will you provide a stimulus for a bug of this sort to show its face? Does the application read data in from the command line? A file? A database? A network?
What would be the equipment? What tools will you use to assist in finding this bug? Maybe a script to create some garbage input. Maybe you can catch problems even earlier by using a code analyzer, or by using different compilers set to be as strict as possible.
Where would an ideal location be? Where would you expect to find this type of bug? At the application’s entry (such as initially reading a file), or deeper in the code (arguments passed between functions)? The approach would be different depending on where you’re fishing (deep sea vs. a freshwater creek).
When would you expect to find these issues? Is there a particular time you might find a crash? Does your application behave differently at certain times of day, or when certain other events are happening?
Hmm… Intriguing. What are you testing right now that could benefit from this approach?
I can’t believe it took me this long to notice and then apply this principle to testing. I mean, if you’re a software tester, how many times have you noticed that sometimes we just run tests, and the tests run and none fail, and everything’s great? It’s great! Awesome! Ship it! Probably plenty, right? We like when tests pass–it means the product may be pretty good. Not perfect, but it’s at least less broke than it was.
But think about this: running a set of tests, and not having particular types of bugs in mind, is like dumping a tackle box into the ocean. If you don’t know what you’re fishing for, how will you be able to catch it? Mind. Equals. Blown.
Hopefully this has helped–consider what kind of bug you’re looking for, and don’t forget your fishing BELT.
Happy fishing!
– Fritz
testzius 
OK, but the stated objective of testing is to ensure that the software meets its requirements (look at ISEB for confirmation).
Bug-hunting is not the main objective! Having said this, yes there are places which are more likely to have bugs- particularly the documentation. So how many projects test the documents? I thought so. When I took this approach at a bank, I found 60 defects (some serious) without even testing a line of code,
LikeLike
Hi Neil, thanks for commenting. I agree that testing requirements is a great way to start–sometimes the people writing the requirements just don’t know what they don’t know.
I’m curious if you can share what kind of defects you found and what was wrong with the requirements? Wondering if I could learn about some examples.
LikeLike
Copy and paste errors, incorrect data specifications, and types mostly.
On a different project, the analyst write “User enters in valid data” – meanig valid data, but the programmers misread it. Also from the same analyst, a field described as ” display only if populated”. He meant display-only, or protected, but the programmers took him at his word and didn’t display it if empty – therefore no-one could put anything in it.
LikeLike
That should read TYPOs of course, not types
LikeLike
Haha, I was about to say, “I see what you did there.” Great examples. I agree that making sure the requirements make sense–even before the devs start coding–is a great way to prevent bugs.
This post is geared toward what’s -not- in the requirements. There are surely places (and likely you’ve seen this in bank applications) where things aren’t specified in the requirements, that end up being bugs. SQL injection is an example. I’ve yet to see a requirement containing this, but if a hacker got a bunch of account info by doing this, nobody would argue it’s a bug.
Testing outside the requirements is actually a big enough topic that I want to write a post about it later.
Again, thanks for commenting. One reason why I made this blog is to learn from other people, and you’re already teaching me about another aspect of testing I hadn’t given a lot of thought on.
LikeLike
Interesting to see I was not the only SQA professional use this comparison. As an avid fishermen and organizer of our 5th annual SQA fishing trip (yes, real fishing starting at 6am), I mentioned the similarities to my Lead yesterday when discussing the thoroughness needed in the preparation, variety, presentation and persistence. As a fisherman or testing professional, you will get less hits if you give up after the first approach before you move on to the next location. Sometimes several casts or attempts need to be made before triggering a hit.
Thanks for sharing.
Tony A
LikeLike