How To Write Unit Tests in English

People like to be able to read tests. It makes it easier to understand, debug and maintain when we don’t have to deal with codey-lookin’ code.

The problem is, the higher level tests, such as end-to-end tests, are the ones that usually have that level of readability. But unit tests are really quick, and can give us fast feedback, but are written in more raw code.

Do you think it’d be helpful to write unit tests in English, like the end-to-end tests can have? If so, you’re in luck. Because today, I’m going to show you an example of how you can combine high readability with unit tests, using cucumber, and some tricks with dynamic programming. We’ll take an iterative approach by building new functionality as we go, while refactoring to make it easier to maintain.

Getting Started

The base language for this post is written in Ruby. Both Ruby and cucumber are easy to install so I won’t get into details here.

First, let’s set up our cucumber project by running ‘cucumber –init’ in a new directory. I called mine “English Unit Testing”. The directory structure will look like this:

English Unit Testing/ 
   features/ 
   step_definitions/
   support/
      env.rb

Next, we’ll add a new directory to hold our classes that we want to unit test against. This can really be anywhere in our file system but I put it in this project so everything’s in one place:

English Unit Testing/ 
   classes/
   features/ 
   step_definitions/
   support/
      env.rb

Our First Test Class

Then we’ll make a simple class that does some math for us, called arithmetic.rb. It contains this:

class Arithmetic
 def add(a, b)
   return a.to_i + b.to_i
 end

 def subtract(a, b)
   return a.to_i - b.to_i
 end

 def multiply(a, b)
   return a.to_i ** b.to_i
 end
end

This is saved in the classes directory we just made:

English Unit Testing/ 
   classes/
      arithmetic.rb
   features/ 
   step_definitions/
   support/
      env.rb

Dynamic Time-Saving

Next we’ll do the first dynamic programming trick. When writing Ruby, if you need your script to know about another file somewhere, you have to “require” it, like this:

require File.expand_path("/path/to/file.rb")

I know that when more classes are created I’m not going to want to go through and put more “require” lines in my code. Mainly because, I’d probably forget to do it 🙂  But what I can do is require everything that’s in a particular folder.

The env.rb file, in the support directory, can hold what cucumber calls “hooks”. Hooks are special steps that we can make run at certain times, such as before running a set of tests.

So I’m going to add a Before hook in env.rb, that goes through the classes directory, and does a “require” on each file found in there:

Before do 
 Dir[File.expand_path("**/classes/*")].each {|file| require file }
end

Good. That’s one bit of maintenance we won’t have to do later. At runtime, our tests will automatically know about arithmetic.rb so we can start using it immediately.

Writing the Feature

Next, we’ll write our first feature file, and stick it in the features/ directory:

English Unit Testing/ 
   classes/
      arithmetic.rb
   features/ 
      test.feature
   step_definitions/
   support/
      env.rb

And we’ll describe the feature like this:

Feature: Unit Test Example Using Cucumber

Scenario: Addition Test
When I call Arithmetic::add with arguments 2 and 3
Then the result should be 5

Scenario: Subtraction Test
When I call Arithmetic::subtract with arguments 5 and 3
Then the result should be 2

Scenario: Multiply Test
When I call Arithmetic::multiply with arguments 2 and 3
Then the result should be 6

When we run this, cucumber will yell about us not having any steps defined, and then helpfully generate some stubs for us. That’s fine. For now we can use those stubs so we can continue, by pasting them directly into steps.rb:

When(/^I call Arithmetic::add with arguments (\d+) and (\d+)$/) do |arg1, arg2|
end

When(/^I call Arithmetic::subtract with arguments (\d+) and (\d+)$/) do |arg1, arg2|
end

When(/^I call Arithmetic::multiply with arguments (\d+) and (\d+)$/) do |arg1, arg2|
end

Then(/^the result should be (.*)$/) do |arg1|
end

Now we’ll flesh out the steps. Each method we test–add, subtract and multiply–does the desired operation and returns a value to us. So all we’re going to do is capture that returned value into a global variable, so that we can use it later:

When(/^I call Arithmetic::add with arguments (\d+) and (\d+)$/) do |arg1, arg2|
 $result = Arithmetic.new.add(arg1, arg2)
end

When(/^I call Arithmetic::subtract with arguments (\d+) and (\d+)$/) do |arg1, arg2|
 $result = Arithmetic.new.subtract(arg1, arg2)
end

When(/^I call Arithmetic::multiply with arguments (\d+) and (\d+)$/) do |arg1, arg2|
 $result = Arithmetic.new.multiply(arg1, arg2)
end

Notice that we can call the Arithmetic class directly. That’s because we took care of that in the Before hook earlier.

Next we add code for the part where we check the results. We should make sure that when we compare our expected result with our actual one, that the data types are the same. So we can convert both variables to strings using the to_s method, and we compare against the global variable $result:

Then(/^the result should be (.*)$/) do |arg1|
 fail "Expected #{arg1}, got #{$result}" if $result.to_s != arg1.to_s
end

Now when we run the test, here’s what we get:

Feature: Unit Test Example Using Cucumber

 Scenario: Addition Test # features/test.feature:3
 When I call Arithmetic::add with arguments 2 and 3 # features/step_definitions/steps.rb:1
 Then the result should be 5 # features/step_definitions/steps.rb:18

 Scenario: Subtraction Test # features/test.feature:7
 When I call Arithmetic::subtract with arguments 5 and 3 # features/step_definitions/steps.rb:5
 Then the result should be 2 # features/step_definitions/steps.rb:18

 Scenario: Multiply Test # features/test.feature:11
 When I call Arithmetic::multiply with arguments 2 and 3 # features/step_definitions/steps.rb:9
 Then the result should be 6 # features/step_definitions/steps.rb:18
 Expected 6, got 8 (RuntimeError)
 ./features/step_definitions/steps.rb:19:in `/^the result should be (.*)$/'
 features/test.feature:13:in `Then the result should be 6'

Failing Scenarios:
cucumber features/test.feature:11 # Scenario: Multiply Test

Hey what the heck?? Why did 2 * 3 turn out to be 8? Let’s go take a look at the multiply method:

def multiply(a, b)
 return a.to_i ** b.to_i
end

Ah there we go. I used a ** operator instead of *. That gave me an exponent instead of multiplying–2 to the power of 3 is 8. See? That’s why we unit test, people!

So I’ll fix it…

def multiply(a, b)
 return a.to_i * b.to_i
end

…and then rerun:

Feature: Unit Test Example Using Cucumber

 Scenario: Addition Test # features/test.feature:3
 When I call Arithmetic::add with arguments 2 and 3 # features/step_definitions/steps.rb:1
 Then the result should be 5 # features/step_definitions/steps.rb:18

 Scenario: Subtraction Test # features/test.feature:7
 When I call Arithmetic::subtract with arguments 5 and 3 # features/step_definitions/steps.rb:5
 Then the result should be 2 # features/step_definitions/steps.rb:18

 Scenario: Multiply Test # features/test.feature:11
 When I call Arithmetic::multiply with arguments 2 and 3 # features/step_definitions/steps.rb:9
 Then the result should be 6 # features/step_definitions/steps.rb:18

3 scenarios (3 passed)
6 steps (6 passed)
0m0.666s

Beauty. All green.

Now it’s time for our first refactoring. The steps we’ve written work fine for now, but they’re tied to a particular class, and a particular method within the class. If we were to continue making classes, we’d have to write more steps. We’d spend a lot of time writing them and then maintaining them. Not optimal.

The steps themselves aren’t much different from each other though. It would be super if we could have a more generic step that treats those two pieces of information as variables.

So let’s see if we can do that:

When(/^I call (.*)::(.*) with arguments (.*) and (.*)$/) do |class_name, method_name, arg1, arg2|
end

Next we’ll flesh out this step.

This is gonna look a little different because we’ll be using some of ruby’s own internal methods to do some introspection. We’re basically going to be treating the class_name and method_name as an actual class, and an actual method:

When(/^I call (.*)::(.*) with arguments (.*)$/) do |class_name, method_name, arg1, arg2|
  $result = eval "#{class_name}.new.#{method_name}(#{arg1}, #{arg2})"
end

What we did with the “eval” call is say, “treat the following string as if it were a piece of code, and then execute it”. It ends up calling the method within the class–whatever those happened to be–and returns a result, which is stored in $result like before.

This means we can get rid of the previous 3 steps, that were tied to Arithmetic, add, subtract and multiply. End result is still the same:

Feature: Unit Test Example Using Cucumber

 Scenario: Addition Test # features/test.feature:3
 When I call Arithmetic::add with arguments 2 and 3 # features/step_definitions/steps.rb:1
 Then the result should be 5 # features/step_definitions/steps.rb:18

 Scenario: Subtraction Test # features/test.feature:7
 When I call Arithmetic::subtract with arguments 5 and 3 # features/step_definitions/steps.rb:5
 Then the result should be 2 # features/step_definitions/steps.rb:18

 Scenario: Multiply Test # features/test.feature:11
 When I call Arithmetic::multiply with arguments 2 and 3 # features/step_definitions/steps.rb:9
 Then the result should be 6 # features/step_definitions/steps.rb:18

3 scenarios (3 passed)
6 steps (6 passed)
0m0.666s

Awesome. Let’s go ahead and write another method that we can immediately write a test for, without writing any new cucumber code:

class Arithmetic
 (...)

   def exponent(a, b)
      return a.to_i ** b.to_i
   end
end
Scenario: Exponent Test
When I call Arithmetic::exponent with arguments 2 and 5
Then the result should be 32
 Scenario: Exponent Test # features/test.feature:15
 When I call Arithmetic::exponent with arguments 2 and 5 # features/step_definitions/steps.rb:18
 Then the result should be 32 # features/step_definitions/steps.rb:23

Cool huh?

This is still pretty useful, but it’s still tied to methods and classes that take in two arguments.

For the next refactoring, we’ll allow for a flexible amount of arguments. We’ll also create another class and method to test against.

First we’ll change the step to expect a string of arguments at the end:

When(/^I call (.*)::(.*) with arguments (.*)$/) do |class_name, method_name, args|
end

We don’t know how the arguments might be supplied. They could look like any of these:

1, 2, 3
1, 2 and 3
1 and 2

What we want to do then is normalize this data down, and then turn it into an array, that we can fling at whatever method we’re testing. For this, we’ll use some internal ruby methods to massage the data down to how we want it to look, and then use the split() method to turn that string into an array:

When(/^I call (.*)::(.*) with arguments (.*)$/) do |class_name, method_name, args|
 args = args.gsub(/, /, ",").gsub(/ and /, ",").split(",")
end

What we did was replace all occurrences of comma-space, and the word “and” with spaces on either side, with just a comma. Then we took that result and split it on commas, to turn it into an array.

Next we’ll make a change to the “eval” call from before, to handle this array of arguments instead of a static list:

When(/^I call (.*)::(.*) with arguments (.*)$/) do |class_name, method_name, args|
 args = args.gsub(/, /, ",").gsub(/ and /, ",").split(",")
 $result = eval "#{class_name}.new.#{method_name}(*#{args})"
end

The “*” is called the “splat operator”. It just means, “this is a list of parameters–I don’t know how many there are so… here ya go.”

Now let’s make another class and method to test this functionality:

English Unit Testing/ 
   classes/
      arithmetic.rb
      statistics.rb
   features/ 
      test.feature
   step_definitions/
   support/
      env.rb
class Statistics
 def average(*args)
   average = 0.0
   args.each do |i|
     average += i.to_i
   end
   return average / args.size.to_f
 end
end

Now we can write a test for this class and method both, in one shot:

Scenario: Average Test
When I call Statistics::average with arguments 1, 2, 3, 4, 5
Then the result should be 3.0
 Scenario: Average Test # features/test.feature:24
 When I call Statistics::average with arguments 1, 2, 3, 4, 5 # features/step_definitions/steps.rb:13
 Then the result should be 3.0 # features/step_definitions/steps.rb:23

So we’ve now got just two steps defined:

When(/^I call (.*)::(.*) with arguments (.*)$/) do |class_name, method_name, args|
 args = args.gsub(/, /, ",").gsub(/ and /, ",").split(",")
 $result = eval "#{class_name}.new.#{method_name}(*#{args})"
end

Then(/^the result should be (.*)$/) do |arg1|
 fail "Expected result of #{arg1}, got #{$result} instead" if $result.to_s != arg1.to_s
end

…and look at all the testing we can do.

Advantages

Although there are already unit testing tools out there for ruby code, writing tests in this way doesn’t require installing any of them. If you’re already using cucumber for your tests, it can be used for this too.

Plus, you get all the readability that comes with using a BDD tool. When we can understand what’s being tested, it’s easier to know where there might be gaps in coverage.

Next Steps

Some other things that could be done:

  • Use Scenario Outlines to drive your tests with data tables, similar to “Theory” in Java and C# unit testing
  • Track what classes and methods have been tested and throw an error if any were missed
  • Include support for integration testing

The sky’s the limit–what else can you think of?

Hope you had fun learning 🙂

–Fritz

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s