How to Write Unit Tests Which Won't Tie Our Hands

I have written my share of bad unit tests. Sometimes (but not always) those tests were so shallow that they didn't really test anything, and yet, they were very expensive to maintain. For example, it was quite common that making the required changes to the production code took 15 minutes and fixing the unit tests took several hours. Needless to say, no other person wanted to fix our unit tests because it was boring and soul crushing work.

This blog post identifies the problem and describes how we can write unit tests which embrace change (sort of). Let's begin.

The Road to Hell Is Paved With Good Intentions

Before we can understand the problem which makes unit tests very expensive to maintain, we have to define the term unit test. Here are three different definitions which explain what a unit test is:

A unit test is a block of code that verifies the accuracy of a smaller, isolated block of application code, typically a function or method. The unit test is designed to check that the block of code runs as expected, according to the developer’s theoretical logic behind it. The unit test is only capable of interacting with the block of code via inputs and captured asserted (true or false) output.

When a block of code requires other parts of the system to run, you can’t use a unit test with that external data. The unit test needs to run in isolation. Other system data, such as databases, objects, or network communication, might be required for the code’s functionality. If that's the case, you should use data stubs instead. It’s easiest to write unit tests for small and logically simple blocks of code.

What is unit testing - AWS

A unit test is a way of testing a unit - the smallest piece of code that can be logically isolated in a system. In most programming languages, that is a function, a subroutine, a method or property. The isolated part of the definition is important. In his book "Working Effectively with Legacy Code", author Michael Feathers states that such tests are not unit tests when they rely on external systems: “If it talks to the database, it talks across the network, it touches the file system, it requires system configuration, or it can't be run at the same time as any other test."

What Is Unit Testing - SmartBear

Despite the variations, there are some common elements. Firstly there is a notion that unit tests are low-level, focusing on a small part of the software system. Secondly unit tests are usually written these days by the programmers themselves using their regular tools - the only difference being the use of some sort of unit testing framework. Thirdly unit tests are expected to be significantly faster than other kinds of tests.

Unit Test - Martin Fowler

Based on these definitions, there seems to be a consensus that a unit test:

• Tests only a small part of the system.
• Is isolated and doesn't use "external systems".
• Is significantly faster than other tests.

Even though these are all good qualities, they can cause problems when a developer selects the size of the tested unit. Let's take a look at two quotes which explain what a unit is:

A unit can be almost anything you want it to be -- a line of code, a method, or a class. Generally though, smaller is better. Smaller tests give you a much more granular view of how your code is performing. There is also the practical aspect that when you test very small units, your tests can be run fast; like a thousand tests in a second fast.

What Is Unit Testing - SmartBear

So there's some common elements, but there are also differences. One difference is what people consider to be a unit. Object-oriented design tends to treat a class as the unit, procedural or functional approaches might consider a single function as a unit. But really it's a situational thing - the team decides what makes sense to be a unit for the purposes of their understanding of the system and its testing.

Unit Test - Martin Fowler

It seems that we can choose the size of the tested unit freely as long as it makes sense, and our team members agree or can at least live with the decision. And yet, there are so many developers (I was one of them) who write unit tests which are extremely expensive to maintain by:

• Choosing a unit that's as small as possible.
• Isolating the system under test from its dependencies by overusing test doubles.

For example, let's assume that we have to write unit tests for Spring MVC REST API endpoints which provide CRUD operations for an entity. The architecture of the system under test looks as follows:

If we use units which are as small as possible and isolate the dependencies of these units by using test doubles, we end up with two units whose dependencies are replaced with test doubles. The following image illustrates this situation:

The problem of this approach is that our unit tests are tightly coupled with the implementation details of the system under test. This limits our ability to make changes to our application because these changes tend to break our unit tests which are expensive to fix. For example:

1. We cannot make breaking changes to our REST API endpoint because these changes will break the unit tests which test the changed REST API endpoint.

2. We cannot make any meaningful changes to the implementation of our controller class (like use a totally different service) because its unit tests expect that predefined interactions happen between the system under test and a specific service.

3. We cannot change the API of our service class because:

• This will break the unit tests which test the controller methods which use the changed service methods. The problem is that these unit tests expect that predefined interactions happen between the system under test and our service.
• This will break the unit tests which test the changed service methods.

4. We cannot make any meaningful changes to the implementation of our service class (like use a new service, stop using a service, or use a totally different repository) because its unit tests expect that predefined interactions happen between the system under test and specific services and repositories.

5. We cannot change the API of our repository class because this will break the unit tests which test the service methods which use the changed repository methods. The problem is that these unit tests expect that predefined interactions happen between the system under test and our repository.

The following image identifies the components which can be changed without breaking our unit tests:

It's quite easy to see that the current situation doesn't look good. Let's see how we can make it better.

Increasing the Size of the Tested Unit

If we want to write unit tests which are easier to maintain, we should use units which are as large as possible. We can increase the size of the tested unit by following this rule:

If the component interacts with an external system such as a database, we should replace it with a test double when we configure the system under test. Otherwise, we should use the real component.

If we follow this rule when we write unit tests for our REST API endpoints, we end up with one unit that looks as follows:

This improves our situation and gives us more freedom to make changes to the system under test. For example:

1. We cannot make breaking changes to our REST API endpoint because these changes will break the unit tests which test the changed REST API endpoint.

2. We can change the implementation of our controller class as long as we don't make any changes to the predefined interactions which must happen between the system under test and our repository.

3. We can change the API of our service class as long as we don't make any changes to the predefined interactions which must happen between the system under test and our repository.

4. We can change the implementation of our service class as long as we don't make any changes to the predefined interactions which must happen between the system under test and our repository.

5. We cannot change the API of our repository class because this will break the unit tests of the controller methods which use services that invoke the changed repository methods. The problem is that these unit tests expect that predefined interactions happen between the system under test and our repository.

The following image identifies the components which can be changed without breaking our unit tests:

Next, we will take a closer look at the pros and cons of our new approach.

There Is No Silver Bullet

First of all, this approach isn't a silver bullet and it doesn't give us free hands to change our production code as we see fit (without breaking our unit tests). If we want to write tests which don't care about the implementation details of the system under test, we have to do black-box testing and write integration tests (or some other tests like API tests or end-to-end tests). That being said, this approach makes our work a bit easier because our test code uses less test doubles and that's why:

• Our unit tests are easier and faster to fix if we change the API of the components which are replaced with test doubles and/or make changes to the predefined interactions which must happen between the system under test and our test doubles.
• We can make some changes to the system under test without breaking our unit tests.

This approach isn't perfect and it has two drawbacks:

• If we modify the "structure" of the system under test (introduce a new service, remove a service, and so on), we have to update the setup code which configures the system under test even if we don't make any changes to the predefined interactions which must happen between the system under test and our test doubles.
• Because we replace a component with a test double if it interacts with an external system, the configuration of our test doubles might not be obvious if the interaction happens "too many" layers below the entry point of our test.

In other words, when we decide if we should increase the size of the tested unit, we must understand that it isn't always a good idea. For example, if the system under test has a lot of complex business logic, it's often best to use as small unit size as possible because this simplifies the required setup code and ensures that our unit tests are as easy to write, read, and maintain as possible.

On the other hand, if the code of the system under test is somewhat simple, we should increase the size of the tested unit because it helps us to write unit tests which are easier and faster to write and maintain than unit tests which use the "traditional" unit size (1).

At this point, we should understand why traditional unit tests can be painful to maintain, know how we can make that pain tolerable by increasing the size of the tested unit, and understand when we should use this new technique. Let's summarize what we learned from this blog post.

Summary

This blog post has taught us three things:

• If we use the traditional unit size and replace the dependencies of every class with test doubles, we cannot make changes to our implementation because these changes break our unit tests.
• If the system under test has a lot of complex business logic, we should use as small unit size as possible because this simplifies the required setup code and ensures that our unit tests are as easy to write, read, and maintain as possible.
• If the code of the system under test is somewhat simple, we should increase the size of the tested unit because it helps us to write unit tests which are easier and faster to write and maintain than unit tests which use the "traditional" unit size (1).