Unit test Exceptions

Testing the “happy path” of the code, when everything goes right, is fine, however error handling and Exceptions is just as much a part of the code under test. In fact, it is possibly a more delicate area, since you want an application which degrades gracefully in the event of error. This post goes through different ways of setting expectations on thrown Exceptions.

In the first method below, the old style pre-Junit 4 way of asserting for an Exception is shown. The idea is that the Exception must be thrown, so if the execution reaches the fail() statement, that did not happen. The expected path is instead that the catch-block engages, with the expected Exception type. If a different type of Exception, which is not a sub-class of the caught Exception is thrown, it propagates out of the method and the test fails. Although this style is a bit clunky and verbose, it has a few advantages over the annotation-style: You have control over exactly what point in the code you expect the Exception to be thrown; you can inspect the Exception and assert its message; you can set a custom error message.

  public void testOldStyle() {
    try {
      Integer.parseInt(INVALID_INTEGER);
      fail("Expected an Exception to be thrown");
    } catch (NumberFormatException e) {
      assertEquals("For input string: \"" + INVALID_INTEGER + "\"", e.getMessage());
    }
  }

Since Java 6 and Junit 4, annotations become available, and the @Test annotation is now the way to declare a test method. It comes with an extra parameter expected which takes a Class type indicating which Exception is expected to be thrown. This approach is clean and even minimalist. If all there is to a test is a one-liner like shown below, this is a perfectly fine way to declaring the expectation. However, it loses the ability to inspect the message or cause. Furthermore, if the test method contains more lines, there is no way to control or verify from where the Exception originated.

  @Test(expected = NumberFormatException.class)
  public void annotation() {
    Integer.parseInt(INVALID_INTEGER);
  }

Enter JUnit 4.7, and the @Rule annotation and ExpectedException rule. It solves the problem with the Test annotation above, but retains a clean way of expressing the assertion. In the example below, the Rule annotation makes sure the thrown field is initialised a-new before every test method. It can then be used right above the method which is expected to throw an Exception, and can assert on its type and message. The expectMessage() method asserts that the message contains rather than equals the expected string.

  @Rule
  public final ExpectedException thrown = ExpectedException.none();

  @Test
  public void testThrown() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(INVALID_INTEGER);

    Integer.parseInt(INVALID_INTEGER);
  }

The ExpectedException class comes with some extra assertion methods which takes the popular Hamcrest matchers. In the code below, the endsWith matcher is used.

import static org.hamcrest.CoreMatchers.endsWith;

  @Test
  public void hamcrest() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(endsWith(INVALID_INTEGER + "\""));

    Integer.parseInt(INVALID_INTEGER);
  }

Finally, the expectCause() method takes another Hamcrest matcher, where for example the type of the contained cause can be asserted. Notice that the outer exception type can be asserted as well. The expectCause assertion only goes one level deep, so if further unnesting is required, a custom Matcher could be implemented. In this example, the wrapping RuntimeException does not alter the message, so it can be asserted directly. If the outer Exception has its own message, another custom Matcher would be needed to assert on the inner message.

import static org.hamcrest.CoreMatchers.isA;

  @Test
  public void cause() {
    thrown.expect(RuntimeException.class);
    thrown.expectMessage(INVALID_INTEGER);
    thrown.expectCause(isA(NumberFormatException.class));

    try {
      Integer.parseInt(INVALID_INTEGER);
    } catch (NumberFormatException e) {
      throw new RuntimeException(e);
    }
  }

Here is the full test case listing.

/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.junit;

import static org.hamcrest.CoreMatchers.endsWith;
import static org.hamcrest.CoreMatchers.isA;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class ExceptionsTest {

  private static final String INVALID_INTEGER = "invalid integer";

  @Rule
  public final ExpectedException thrown = ExpectedException.none();

  @Test
  public void testOldStyle() {
    try {
      Integer.parseInt(INVALID_INTEGER);
      fail("Expected an Exception to be thrown");
    } catch (NumberFormatException e) {
      assertEquals("For input string: \"" + INVALID_INTEGER + "\"", e.getMessage());
    }
  }

  @Test(expected = NumberFormatException.class)
  public void annotation() {
    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void testThrown() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(INVALID_INTEGER);

    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void hamcrest() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(endsWith(INVALID_INTEGER + "\""));

    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void cause() {
    thrown.expect(RuntimeException.class);
    thrown.expectMessage(INVALID_INTEGER);
    thrown.expectCause(isA(NumberFormatException.class));

    try {
      Integer.parseInt(INVALID_INTEGER);
    } catch (NumberFormatException e) {
      throw new RuntimeException(e);
    }
  }
}