desert

Testing Without Requirements

So, this is a question that comes up a lot: what do we do when we start a job, or a project, and we need to test, but there aren’t any requirements?

To some of us, it might be hard to see how we can be put in this position.

If you’ve only ever been at places where they have documented test cases or specs on a system, then the idea might be foreign.

But there are some places that are either too busy to write requirements, or just don’t see the need for them.

There are lots or reasons why a team or company ends up at this point–some good reasons, some bad ones.

But regardless what the reasons are, there may come a time where you’re in this position, and you’ll be asked to test against something that doesn’t have requirements written, and maybe… noooooooobody knows fully how it works.

Today I’ll explain how it’s not an impossible task–that you can not only succeed, but thrive in this case, and sharpen your testing skills too.

red-green
I’m a tester… and I can test without requirements… if I have to… I guess.

 

Black Box Testing

When we take a piece of software, and we don’t really know what’s going on inside it–“inside the box” so to speak–we’re using Black Box Testing to test it.

If you’re not familiar with the term, Black Box might sound kinda spooky.

It’s really not.

It’s a little like… well, it’s like picking locks.

You can’t really see inside the lock, but you can use tools, and your perception, to feel around and learn what’s in there. And, you use what you know about previous locks, to glean information about this one.

Same thing with Black Box Testing.

Probably you’ve tested software before, and you’ve found bugs too. We’ll use these tools and perception for future testing.

Below we have an example of a simple form, along with a sample conversation that leads up to you having to test something without requirements.

Check out this form:

advanced-form

And then here’s a hypothetical conversation you might have with the manager:

Hey so yeah, we had an intern throw together a simple form for entering customer data. We were needing it for a client POC. It was good experience for her.

Client loved it, so we went ahead and built the rest of the product after the fact. This part, this form, isn’t customer facing–we just wanted to have a hook to load customer records into the system.

Aaaaaand now the intern’s gone, and we just never took the time to spec out such a simple form. But it still has to be tested. ¯\_(ツ)_/¯

Awesome. No requirements. Perfect.

Don’t lose hope yet though, because there’s still a lot we can test for. We’re not flying blind.

We know that this form likely stores data in a database somewhere, and we can cover a lot of tests just by seeing if our data gets transferred over to the database correctly.

Here are some other things we can test:

  • Do the required fields actually enforce that you put something in? What if you leave all but one blank for each test, do they each fail as expected?
  • If this is storing data in a database, can data get “clipped”? If you put in a 20 character name, but the database only stores 18 characters, is that a problem?
  • Does the form error out if not enough or too much info is provided?
  • That date form–what format does it expect? Does it error out appropriately if the wrong format is put in?
  • Does the datepicker next to it populate the date field as expected? Does your locale (US vs. non-US) affect the date format, and does the rest of the system handle it?
  • What if you pick a date in the future, is that bad?
  • What’s the Active checkbox for? Does that state carry over to the database–does a flag get set? Or if you select it does nothing happen?
  • What if you try to make another customer record with the same name, does it stomp the old record or create a new one? And then which of the two is supposed to be used elsewhere in the system?
  • What’s supposed to be in the Investments dropdown? Is that populated from somewhere? Do the contents match the source of the information?
  • What about the email field, does it sanitize to make sure a valid email address is in there? What if you try to splice two addresses together with a “;” character? Does it allow it?
  • For each text field, can you get an XSS or SQL injection attack through? If so, very very bad. That needs to be fixed.
  • Can special characters be handled? Not every name is just letters.
  • When you click OK does the record actually get stored as-is in the database as expected?
  • If you click Cancel then does the data get dropped? If you go back to the form again after canceling, are there any artifacts left over from the previous attempt–fields that were filled in from last time, still showing up next time?
  • Can you modify the DOM and get wonky data shoved in somewhere it doesn’t belong?

There’s a LOT that can be tested. There’s likely more. Can you think of other cases?

A small part of software testing is making sure the software does what it’s supposed to. The remaining, larger part of it, is ensuring that it does NOT do what it’s NOT supposed to do.

It’s by far the bigger challenge, because we have to think of things that other people usually don’t think of.

And, it’s the kind of challenge that will sharpen your skills, and put you a level above people who rely on (and need) requirements in order to function.


If you’re up to the challenge of looking hard at how the testing and automation piece is done, I can help with that. And then I can help you level up. www.archdevops.com

 

10 thoughts on “Testing Without Requirements

  1. Good thoughts! In some ways I think it might even be easier to test without requirement. Sometime we forget that requirements are just someone else’s ‘test plan.’ They are just one view on what the product is supposed to do and can constrain you into thinking only around what does instead of (as you pointed out) what it shouldn’t do.

    Even if we had a requirements doc, I would want to approach it critically as well. Where are the requirements wrong or showing bias?

    Liked by 1 person

  2. Is there any chance you’re confusing “requirements” with requirements documents here?

    you’ll be asked to test against something that doesn’t have requirements written, and maybe… noooooooobody knows fully how it works.

    That’s true. That’s always true, even when there are requirements documents. Nobody knows fully how something works (and, perhaps more significantly, how something doesn’t work) even after we’ve tested. We test in order to find out empirically more about than we knew before. That’s because testing is not only about making sure that the product conforms to requirements (whether written or not). Testing is also about discovering what people want, even when they’re fuzzy on that themselves—and they’re always fuzzy about what they want to some degree.

    In other words, requirements are not merely gathered, and not merely written. Requirements are explored and developed, as we try to build the product and as we learn about it. Testing is an important part of that process.

    You’ve got some excellent ideas for exploration here. They fit very nicely with the Honest Manual Writer Heuristic (http://www.developsense.com/blog/2016/05/the-honest-manual-writer-heuristic/)

    Cheers,

    —Michael B.

    Liked by 1 person

    1. Might’ve appeared that way, but I hadn’t made the distinction, mentally, between written and non-written requirements.

      I agree with you, and I think we’re on the same page–just I goofed on the verbiage here 🙂

      Like

  3. If you do not know what the system is required to do how can you say that it functions properly?
    So either you are not doing your job or you are using the wrong word.

    Like

    1. Fair question. Short answer: black box testing.

      We can know what parts of software are supposed to do, just by knowing what they are. Login pages, data tables, webservices, all have things they should (or shouldn’t) do.

      We can still make hypotheses against these, and test them, and get good info about whether the software’s OK, even without a set of requirements to test against.

      Like

  4. In Genera,l we can do a “blind” testing without any docs/requirements. But if we work on a project where you work with a team of developers its kinds difficult to work without requirements because I always have debates with devs that all tickets which I rised are not a bugs)) But eventually we decide that 90% of them are really issues. But it takes a time to tell them that this is should works in another way because its not logical and usable. Thats why its better to spend time on requirements.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s