Archive for the Category 'junit'

An Alternative Approach for Grails Integration Tests

Tuesday, December 30th, 2008

I like the approach that Grails takes for integration (and unit) testing; it’s similar to the testing strategy that I’ve used in non-Grails apps. Using HSQLDB to create an in-memory test database significantly reduces the time required to run database tests, and Hibernate’s database independence means you don’t need to pollute application code to make it testable.

Thomas Mueller originally wrote Hypersonic SQL but he ended up discontinuing it, and then other developers picked up where he left off, creating HSQLDB. For the past few years Mueller has been developing H2 from scratch and it has benefited from hindsight and lessons learned from Hypersonic and HSQLDB. According to the HSQLDB website there are some cool new features coming in 1.9 and 2.0, but H2 is already significantly more advanced. So I’ve chosen to use H2 here, but it would be pretty much the same with HSQLDB – just change the configuration properties.


There are many options for database testing; the approach Grails takes is to start a transaction for each test and roll it back afterwards. The approach I’d previously used was to rebuild the database between tests. Since the database is in-memory this is very fast and allows what I think is a more natural coding style. Since each test starts with a guaranteed clean database, there’s no need for a single wrapper transaction – I run everything in autocommit mode, similar to the way application code works in the app. If transactional behavior is required, just group multiple statements in a transaction – Grails makes it simple with withTransaction blocks and with transactional Services.

The key to this approach is a base test class that extends GroovyTestCase and which all Domain class test classes (or any test that accesses the database) extend:

import groovy.sql.Sql

import org.apache.log4j.Logger
import org.codehaus.groovy.grails.orm.hibernate.cfg.DefaultGrailsDomainConfiguration
import org.hibernate.cfg.Configuration

import org.hibernate.tool.hbm2ddl.SchemaExport

/**
 * Abstract base class for H2-based integration tests.
 * Rebuilds an in-memory database for each test.
 */
abstract class AbstractIntegrationTestCase extends GroovyTestCase {

  private static Configuration _configuration

  /** Disable Grails transaction around the tests. */
  def transactional = false

  def dataSource
  def grailsApplication
  def messageSource
  def sessionFactory

  /**
   * Replace the base class logger with one that logs using the actual class name.
   */
  protected final Logger log = Logger.getLogger(getClass())

  /**
   * {@inheritDoc}
   * @see junit.framework.TestCase#setUp()
   */
  @Override
  protected void setUp() {
    super.setUp()

    if (!_configuration) {
      // 1-time creation of the configuration

      Properties properties = new Properties()
      properties.setProperty 'hibernate.connection.driver_class', 'org.h2.Driver'
      properties.setProperty 'hibernate.connection.username', 'sa'
      properties.setProperty 'hibernate.connection.password', ''
      properties.setProperty 'hibernate.connection.url', 'jdbc:h2:mem:test'
      properties.setProperty 'hibernate.dialect', 'org.hibernate.dialect.H2Dialect'

      _configuration = new DefaultGrailsDomainConfiguration(
        grailsApplication: grailsApplication,
        properties: properties)
    }

    // rebuild the database before each test.
    new SchemaExport(_configuration).create(false, true)
    loadTestData dataSource

    clearSession()
  }

  protected void loadTestData(dataSource) {
    // override as necessary; by default run GORM create statements,
    // but could also load sql script(s)
  }

  /**
   * Run a JDBC query independent of Hibernate to verify the table count.
   * @param table  the table to query
   * @param where  optional WHERE clause
   * @return  the count
   */
  protected int findTableCount(String table, String where = null)  {

    String sqlSelect = "SELECT COUNT(0) FROM " + table
    if (where) {
      sqlSelect += " WHERE " + where
    }

    Sql.newInstance(dataSource).firstRow(sqlSelect)[0]
  }

  /**
   * Save or create the bean if it validates, and log validation errors otherwise.
   * @param bean  the bean
   * @param flush  if <code>true</code>, flush on save/update
   * @param throwException  if <code>true</code>, throw an exception if
   *    validation fails instead of returning null
   * @return  the saved/updated instance or <code>null</code>
   *    if there was a problem
   */
  protected def saveOrUpdate(bean, flush = false, throwException = true) {
    bean.validate()
    if (bean.hasErrors()) {
      println "problem creating/updating bean: ${bean}"
      def locale = Locale.getDefault()
      bean.errors.each { fieldErrors ->
        fieldErrors.allErrors.each { error ->
          println messageSource.getMessage(error, locale)
        }
      }
      if (throwException) {
        fail 'validation failed in saveOrUpdate'
      }
      return null
    }

    return bean.save(flush: flush)
  }

  protected void clearSession() {
    sessionFactory.currentSession.clear()
  }

  protected def getDomainClass() {
    return grailsApplication.getArtefact('Domain', getDomainClassName())
  }

  protected String getDomainClassName() {
    return getClass().name - 'Tests'
  }
}

It sets transactional = false to keep Grails from starting a transaction around each test. There’s a static variable to ensure that the one-time Configuration creation runs just once for all tests, and it runs SchemaExport.create() to run the drop and create DDL statements to create new empty tables for each test. A protected loadTestData() method is used to populate test data. Individual tests might need specific data but often there’s a core set of data that all tests use – this is good place to populate that shared data and to override as necessary.

Other utility methods include findTableCount() (with an optional WHERE clause) to check table row counts independent of Hibernate, saveOrUpdate() to save or update Domain class instances and log validation errors to help debug data population code issues, and clearSession() to clear the Hibernate Session’s 1st-level cache (to force reloading saved instances).

Here’s some sample test code from a test class:

void testWhatever() {

   Thing.withTransaction { status ->
      saveOrUpdate new Thing(...)
      saveOrUpdate new OtherThing(...)
      ...
   }

   clearSession()

   assertEquals ...
   assertTrue ...
}

but transactions aren’t required:

void testSomethingElse() {

   saveOrUpdate new Thing(...)
   assertEquals 5, Thing.count()
   assertTrue ...
}

For the most part I find that the test code I write using this isn’t much different from what I’d write in a regular Grails app. But it is definitely more flexible, and I find it’s more intuitive.

An Updated JUnit AllTests Suite

Thursday, December 13th, 2007

I upgraded to JUnit 4.4 and my AllTests utility class that runs all tests in a project broke because I’d been using internal APIs (org.junit.internal) that were refactored between 4.1 and 4.4. It took some digging into their source code but I’ve got it fixed.

Of course Ant has great support for JUnit from the command line – I use this in Eclipse to allow quick “Run As JUnit Test” support, set breakpoints, take advantage of the IDE’s UI that monitors progress, etc.

This is an extension of their @RunWith/@SuiteClasses annotation facility that auto-discovers all test classes rather than specifying a list of class names. A hard-coded list is a pain to maintain and for a large project would be enormous (assuming decent test coverage that is …).

The old code had a static inner class that extended org.junit.internal.runners.TestClassRunner and passed in the results of a directory scan of *Test.class files that aren’t abstract, rather than using the value of the @SuiteClasses annotation. The current implementation is very similar – it extends org.junit.runners.Suite instead.

One interesting feature of this approach is that you have a hook into the start of the test suite, and can also attach event listeners to be notified of start/stop events, failures, etc. This is important because test classes no longer extend a JUnit class – annotations are used instead. So for example it’s no longer possible for to access the name of the currently running test method in a generic fashion.

You could add code to each method but this is brittle and clutters the code. Instead of overriding setUp we use @Before annotatons on one or more methods, but these aren’t the test methods and are invoked using reflection so there’s no hook into the test method that’s about to be run. So in the code below I add an event listener to log the start and end of each test method. This was useful recently when I was chasing down a Collection leak that caused tests to hang once the connection pool was maxed out.

To use this, just right-click it in the class tree in Eclipse and click “Run As JUnit Test”.

import java.io.File;
import java.io.UnsupportedEncodingException;
import java.lang.reflect.Modifier;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import org.apache.log4j.Logger;
import org.junit.internal.runners.InitializationError;
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.Suite;

/**
 * Discovers all JUnit tests and runs them in a suite.
 */
@RunWith(AllTests.AllTestsRunner.class)
public final class AllTests {

  private static final File CLASSES_DIR = findClassesDir();

  private AllTests() {
    // static only
  }

  /**
   * Finds and runs tests.
   */
  public static class AllTestsRunner extends Suite {

    private final Logger _log = Logger.getLogger(getClass());

    /**
     * Constructor.
     *
     * @param clazz  the suite class - <code>AllTests</code>
     * @throws InitializationError  if there's a problem
     */
    public AllTestsRunner(final Class<?> clazz) throws InitializationError {
      super(clazz, findClasses());
    }

    /**
     * {@inheritDoc}
     * @see org.junit.runners.Suite#run(org.junit.runner.notification.RunNotifier)
     */
    @Override
    public void run(final RunNotifier notifier) {
      initializeBeforeTests();

      notifier.addListener(new RunListener() {
        @Override
        public void testStarted(final Description description) {
          if (_log.isTraceEnabled()) {
            _log.trace("Before test " + description.getDisplayName());
          }
        }

        @Override
        public void testFinished(final Description description) {
          if (_log.isTraceEnabled()) {
            _log.trace("After test " + description.getDisplayName());
          }
        }
      });

      super.run(notifier);
    }

    private static Class<?>[] findClasses() {
      List<File> classFiles = new ArrayList<File>();
      findClasses(classFiles, CLASSES_DIR);
      List<Class<?>> classes = convertToClasses(classFiles, CLASSES_DIR);
      return classes.toArray(new Class[classes.size()]);
    }

    private static void initializeBeforeTests() {
      // do one-time initialization here
    }

    private static List<Class<?>> convertToClasses(
        final List<File> classFiles, final File classesDir) {

      List<Class<?>> classes = new ArrayList<Class<?>>();
      for (File file : classFiles) {
        if (!file.getName().endsWith("Test.class")) {
          continue;
        }
        String name = file.getPath().substring(classesDir.getPath().length() + 1)
          .replace('/', '.')
          .replace('\\', '.');
        name = name.substring(0, name.length() - 6);
        Class<?> c;
        try {
          c = Class.forName(name);
        }
        catch (ClassNotFoundException e) {
          throw new AssertionError(e);
        }
        if (!Modifier.isAbstract(c.getModifiers())) {
          classes.add(c);
        }
      }

      // sort so we have the same order as Ant
      Collections.sort(classes, new Comparator<Class<?>>() {
        public int compare(final Class<?> c1, final Class<?> c2) {
          return c1.getName().compareTo(c2.getName());
        }
      });

      return classes;
    }

    private static void findClasses(final List<File> classFiles, final File dir) {
      for (File file : dir.listFiles()) {
        if (file.isDirectory()) {
          findClasses(classFiles, file);
        }
        else if (file.getName().toLowerCase().endsWith(".class")) {
          classFiles.add(file);
        }
      }
    }
  }

  private static File findClassesDir() {
    try {
      String path = AllTests.class.getProtectionDomain()
        .getCodeSource().getLocation().getFile();
      return new File(URLDecoder.decode(path, "UTF-8"));
    }
    catch (UnsupportedEncodingException impossible) {
      // using default encoding, has to exist
      throw new AssertionError(impossible);
    }
  }
}

Fun with EasyMock

Monday, August 13th, 2007

I’m a big fan of Test-driven development and have been using JUnit for 6 or 7 years (since around v3.4 or v3.5). I’ve written lots of mock objects (from simplistic dummy concrete classes that implement an entire interface, to Proxy-based mocks, to a pretty decent mock JMS library) and played with various mock libraries, but the one I’m sticking with and using almost exclusively is EasyMock.

Reinventing a wheel is great for learning, and I’ve done it a lot. But perhaps in a sign that I’m getting older, I’m more concerned with getting stuff done, so now I tend to lean towards existing solutions when good ones exist. I searched for what people tend to use for mock objects and jMock and EasyMock are popular. I prefer the method-based approach of EasyMock to the string-based approach of jMock, since code changes will cause the tests to break immediately, and the mocks will be refactored along with the rest of the code if I use the IDE’s refactoring features.

The general pattern with creating mocks for a tier of your application is that you’re trusting that that tier is tested and works correctly, so it’s appropriate to have mocks return hard-coded values. We’re testing that the caller of the mocked interface does the right thing with correct (or incorrect) inputs, so mocks help us to focus on what’s being tested in isolation.

I’d always found it difficult to unit test Spring MVC controllers, since it seems like they need a running container, and I don’t want to deal with the hassle of setting up Cactus. Plus it seems somewhat useless to test controllers, since they shouldn’t be doing much that isn’t tested in proper automated functional tests. If they do too much business logic, they should be refactored to push that work out to the services layer, right?

The reality is that we do too much business logic in controllers, and usually it’s lazyness but often it’s being pragmatic. We also tend to often do even more “illegal” activities in controllers, such as writing directly to the response rather than delegating to a View. This is usually the case for XML and JSON generated for Ajax responses. So until these get refactored, they need to be tested.

It’s helpful to have partially-functioning servlet container classes such as HttpServletRequest and HttpServletResponse, and the Spring Mock library (spring-mock.jar) is great for that. But for creating mocks for DAOs, services, and other interface-based dependency-injected resources, EasyMock greatly simplifies things. And with their user-contributed Class Extension, you can even mock out concrete classes.

The syntax takes a while to get used to. Basically the flow that I tend to use is:

  1. Create a mock
  2. call expect(mock.[method call]).andReturn([result]) for each expected call
  3. call mock.[method call], then EasyMock.expectLastCall() for each expected void call
  4. call replay(mock) to switch from “record” mode to “playback” mode
  5. inject the mock as needed
  6. call the test method
  7. call verify(mock) to assure that all expected calls happened

As simple as that may sound, it can look very weird at first. Say I have an interface:

public interface Thing {
   void doSomething(String parameter);
   int doSomethingElse(String parameter, int otherParameter);
}

I can create a mock for it via:

Thing thing = EasyMock.createMock(Thing.class);

Then I can register expected calls via

EasyMock.expect(thing.doSomethingElse("woot", 5)).andReturn(123);
EasyMock.expect(thing.doSomethingElse("fubar", 45)).andReturn(321);

thing.doSomething("p");
EasyMock.expectLastCall();

This says that in my calling code, when the client calls thing.doSomethingElse("woot", 5) to return 123, when the client calls thing.doSomethingElse("fubar", 45) to return 321, and to expect a call to thing.doSomething("p").

I can then inject this mock into the class being tested in place of the real one, trusting that it will return known results. I can then concentrate on assuring that the tested class does the right thing.

Thanks to EasyMock, my productivity for testing services and controllers is way up – and my code coverage percentages are too.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.