An Updated JUnit AllTests Suite

I upgraded to JUnit 4.4 and my AllTests utility class that runs all tests in a project broke because I’d been using internal APIs (org.junit.internal) that were refactored between 4.1 and 4.4. It took some digging into their source code but I’ve got it fixed.

Of course Ant has great support for JUnit from the command line – I use this in Eclipse to allow quick “Run As JUnit Test” support, set breakpoints, take advantage of the IDE’s UI that monitors progress, etc.

This is an extension of their @RunWith/@SuiteClasses annotation facility that auto-discovers all test classes rather than specifying a list of class names. A hard-coded list is a pain to maintain and for a large project would be enormous (assuming decent test coverage that is …).

The old code had a static inner class that extended org.junit.internal.runners.TestClassRunner and passed in the results of a directory scan of *Test.class files that aren’t abstract, rather than using the value of the @SuiteClasses annotation. The current implementation is very similar – it extends org.junit.runners.Suite instead.

One interesting feature of this approach is that you have a hook into the start of the test suite, and can also attach event listeners to be notified of start/stop events, failures, etc. This is important because test classes no longer extend a JUnit class – annotations are used instead. So for example it’s no longer possible for to access the name of the currently running test method in a generic fashion.

You could add code to each method but this is brittle and clutters the code. Instead of overriding setUp we use @Before annotatons on one or more methods, but these aren’t the test methods and are invoked using reflection so there’s no hook into the test method that’s about to be run. So in the code below I add an event listener to log the start and end of each test method. This was useful recently when I was chasing down a Collection leak that caused tests to hang once the connection pool was maxed out.

To use this, just right-click it in the class tree in Eclipse and click “Run As JUnit Test”.

import java.io.File;
import java.io.UnsupportedEncodingException;
import java.lang.reflect.Modifier;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import org.apache.log4j.Logger;
import org.junit.internal.runners.InitializationError;
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.Suite;

/**
 * Discovers all JUnit tests and runs them in a suite.
 */
@RunWith(AllTests.AllTestsRunner.class)
public final class AllTests {

  private static final File CLASSES_DIR = findClassesDir();

  private AllTests() {
    // static only
  }

  /**
   * Finds and runs tests.
   */
  public static class AllTestsRunner extends Suite {

    private final Logger _log = Logger.getLogger(getClass());

    /**
     * Constructor.
     *
     * @param clazz  the suite class - <code>AllTests</code>
     * @throws InitializationError  if there's a problem
     */
    public AllTestsRunner(final Class<?> clazz) throws InitializationError {
      super(clazz, findClasses());
    }

    /**
     * {@inheritDoc}
     * @see org.junit.runners.Suite#run(org.junit.runner.notification.RunNotifier)
     */
    @Override
    public void run(final RunNotifier notifier) {
      initializeBeforeTests();

      notifier.addListener(new RunListener() {
        @Override
        public void testStarted(final Description description) {
          if (_log.isTraceEnabled()) {
            _log.trace("Before test " + description.getDisplayName());
          }
        }

        @Override
        public void testFinished(final Description description) {
          if (_log.isTraceEnabled()) {
            _log.trace("After test " + description.getDisplayName());
          }
        }
      });

      super.run(notifier);
    }

    private static Class<?>[] findClasses() {
      List<File> classFiles = new ArrayList<File>();
      findClasses(classFiles, CLASSES_DIR);
      List<Class<?>> classes = convertToClasses(classFiles, CLASSES_DIR);
      return classes.toArray(new Class[classes.size()]);
    }

    private static void initializeBeforeTests() {
      // do one-time initialization here
    }

    private static List<Class<?>> convertToClasses(
        final List<File> classFiles, final File classesDir) {

      List<Class<?>> classes = new ArrayList<Class<?>>();
      for (File file : classFiles) {
        if (!file.getName().endsWith("Test.class")) {
          continue;
        }
        String name = file.getPath().substring(classesDir.getPath().length() + 1)
          .replace('/', '.')
          .replace('\\', '.');
        name = name.substring(0, name.length() - 6);
        Class<?> c;
        try {
          c = Class.forName(name);
        }
        catch (ClassNotFoundException e) {
          throw new AssertionError(e);
        }
        if (!Modifier.isAbstract(c.getModifiers())) {
          classes.add(c);
        }
      }

      // sort so we have the same order as Ant
      Collections.sort(classes, new Comparator<Class<?>>() {
        public int compare(final Class<?> c1, final Class<?> c2) {
          return c1.getName().compareTo(c2.getName());
        }
      });

      return classes;
    }

    private static void findClasses(final List<File> classFiles, final File dir) {
      for (File file : dir.listFiles()) {
        if (file.isDirectory()) {
          findClasses(classFiles, file);
        }
        else if (file.getName().toLowerCase().endsWith(".class")) {
          classFiles.add(file);
        }
      }
    }
  }

  private static File findClassesDir() {
    try {
      String path = AllTests.class.getProtectionDomain()
        .getCodeSource().getLocation().getFile();
      return new File(URLDecoder.decode(path, "UTF-8"));
    }
    catch (UnsupportedEncodingException impossible) {
      // using default encoding, has to exist
      throw new AssertionError(impossible);
    }
  }
}

8 Responses to “An Updated JUnit AllTests Suite”

  1. Chris Lowe says:

    Very useful, thanks for sharing!

  2. Adam Brod says:

    This is exactly what I was looking for. I want to be able to create my own test suites dynamically and this makes it much easier.

  3. […] for. You can read about some annotation-based test suites at this site. This guy gave me the exact code I was looking for, maybe someday they will add this feature directly to […]

  4. Perfect!

    I had written something like this for JUnit 3, and was not looking forward to writing it again.

    Thanks for sharing!

  5. Ran Biron says:

    Excellent! Now I can have my single-click-to-run-tests in a multi-module environment.
    Thanks!

  6. Swati Patel says:

    Thanks this was really useful!

  7. Great article!
    Thanks for sharing!
    I’ll try change this code to filter unit and integration tests.
    \o/

  8. William says:

    Hello,

    Perhaps you would like to answer this question so that you get the credit rather than me:

    http://stackoverflow.com/questions/3681663/how-can-i-recursively-find-and-run-all-junit-4-tests-within-eclipse

    Regards,
    Will

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.