1.2.7.1 JUnit Platform Launcher API

One of the prominent goals of JUnit 5 is to make the interface between JUnit and its programmatic clients – build tools and IDEs – more powerful and stable. The purpose is to decouple the internals of discovering and executing tests from all the filtering and configuration that’s necessary from the outside.

JUnit 5 introduces the concept of a Launcher that can be used to discover, filter, and execute tests. Moreover, third party test libraries – like Spock, Cucumber, and FitNesse – can plug into the JUnit Platform’s launching infrastructure by providing a custom TestEngine.

The launcher API is in the junit-platform-launcher module.

An example consumer of the launcher API is the ConsoleLauncher in the junit-platform-console project.

6.1.1. Discovering Tests

Introducing test discovery as a dedicated feature of the platform itself will (hopefully) free IDEs and build tools from most of the difficulties they had to go through to identify test classes and test methods in the past.

Usage Example:

import static org.junit.platform.engine.discovery.ClassNameFilter.includeClassNamePatterns;
import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
import static org.junit.platform.engine.discovery.DiscoverySelectors.selectPackage;

import java.io.PrintWriter;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.TestExecutionListener;
import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherConfig;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;
import org.junit.platform.launcher.listeners.TestExecutionSummary;
import org.junit.platform.reporting.legacy.xml.LegacyXmlReportGeneratingListener;

LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
    .selectors(
        selectPackage("com.example.mytests"),
        selectClass(MyTestClass.class)
    )
    .filters(
        includeClassNamePatterns(".*Tests")
    )
    .build();

Launcher launcher = LauncherFactory.create();

TestPlan testPlan = launcher.discover(request);

There’s currently the possibility to select classes, methods, and all classes in a package or even search for all tests in the classpath. Discovery takes place across all participating test engines.

The resulting TestPlan is a hierarchical (and read-only) description of all engines, classes, and test methods that fit the LauncherDiscoveryRequest. The client can traverse the tree, retrieve details about a node, and get a link to the original source (like class, method, or file position). Every node in the test plan has a unique ID that can be used to invoke a particular test or group of tests.

Clients can register one or more LauncherDiscoveryListener implementations to get insights into events that occur during test discovery via the LauncherDiscoveryRequestBuilder. The builder registers a default listener that can be changed via the junit.platform.discovery.listener.default configuration parameter. If the parameter is not set, test discovery will be aborted after the first failure is encountered.

6.1.2. Executing Tests

To execute tests, clients can use the same LauncherDiscoveryRequest as in the discovery phase or create a new request. Test progress and reporting can be achieved by registering one or more TestExecutionListener implementations with the Launcher as in the following example.

LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
    .selectors(
        selectPackage("com.example.mytests"),
        selectClass(MyTestClass.class)
    )
    .filters(
        includeClassNamePatterns(".*Tests")
    )
    .build();

Launcher launcher = LauncherFactory.create();

// Register a listener of your choice
SummaryGeneratingListener listener = new SummaryGeneratingListener();
launcher.registerTestExecutionListeners(listener);

launcher.execute(request);

TestExecutionSummary summary = listener.getSummary();
// Do something with the TestExecutionSummary.

There is no return value for the execute() method, but you can easily use a listener to aggregate the final results in an object of your own. For examples see the SummaryGeneratingListener and LegacyXmlReportGeneratingListener.

6.1.3. Plugging in your own Test Engine

JUnit currently provides two TestEngine implementations.

• junit-jupiter-engine: The core of JUnit Jupiter.
• junit-vintage-engine: A thin layer on top of JUnit 4 to allow running vintage tests with the launcher infrastructure.

Third parties may also contribute their own TestEngine by implementing the interfaces in the junit-platform-engine module and registering their engine. By default, engine registration is supported via Java’s java.util.ServiceLoader mechanism. For example, the junit-jupiter-engine module registers its org.junit.jupiter.engine.JupiterTestEngine in a file named org.junit.platform.engine.TestEngine within the /META-INF/services in the junit-jupiter-engine JAR.

HierarchicalTestEngine is a convenient abstract base implementation (used by the junit-jupiter-engine) that only requires implementors to provide the logic for test discovery. It implements execution of TestDescriptors that implement the Node interface, including support for parallel execution.

The junit- prefix is reserved for TestEngines from the JUnit TeamThe JUnit Platform Launcher enforces that only TestEngine implementations published by the JUnit Team may use the junit- prefix for their TestEngine IDs.

• If any third-party TestEngine claims to be junit-jupiter or junit-vintage, an exception will be thrown, immediately halting execution of the JUnit Platform.
• If any third-party TestEngine uses the junit- prefix for its ID, a warning message will be logged. Later releases of the JUnit Platform will throw an exception for such violations.

6.1.4. Plugging in your own Post-Discovery Filters

In addition to specifying post-discovery filters as part of a LauncherDiscoveryRequest passed to the Launcher API, by default custom PostDiscoveryFilter implementations will be discovered at runtime via Java’s java.util.ServiceLoader mechanism and automatically applied by the Launcher in addition to those that are part of the request. For example, an example.CustomTagFilter class implementing PostDiscoveryFilter and declared within the /META-INF/services/org.junit.platform.launcher.PostDiscoveryFilter file is loaded and applied automatically.

6.1.5. Plugging in your own Test Execution Listener

In addition to the public Launcher API method for registering test execution listeners programmatically, by default custom TestExecutionListener implementations will be discovered at runtime via Java’s java.util.ServiceLoader mechanism and automatically registered with the Launcher created via the LauncherFactory. For example, an example.TestInfoPrinter class implementing TestExecutionListener and declared within the /META-INF/services/org.junit.platform.launcher.TestExecutionListener file is loaded and registered automatically.

6.1.6. Deactivating Test Execution Listeners

Sometimes it can be useful to run a test suite without certain execution listeners being active. For example, you might have custom a TestExecutionListener that sends the test results to an external system for reporting purposes, and while debugging you might not want these debug results to be reported. To do this, provide a pattern for the junit.platform.execution.listeners.deactivate configuration parameter to specify which execution listeners should be deactivated (i.e. not registered) for the current test run.

Only listeners registered via the ServiceLoader mechanism within the /META-INF/services/org.junit.platform.launcher.TestExecutionListener file can be deactivated. In other words, any TestExecutionListener registered explicitly via the LauncherDiscoveryRequest cannot be deactivated via the junit.platform.execution.listeners.deactivate configuration parameter.

In addition, since execution listeners are registered before the test run starts, the junit.platform.execution.listeners.deactivate configuration parameter can only be supplied as a JVM system property or via the JUnit Platform configuration file (see Configuration Parameters for details). This configuration parameter cannot be supplied in the LauncherDiscoveryRequest that is passed to the Launcher.

Pattern Matching Syntax

Refer to Pattern Matching Syntax for details.

6.1.7. JUnit Platform Reporting

The junit-platform-reporting artifact contains TestExecutionListener implementations that generate test reports. These listeners are typically used by IDEs and build tools. The package org.junit.platform.reporting.legacy.xml currently contains the following implementation.

• LegacyXmlReportGeneratingListener generates a separate XML report for each root in the TestPlan. Note that the generated XML format is compatible with the de facto standard for JUnit 4 based test reports that was made popular by the Ant build system. The LegacyXmlReportGeneratingListener is used by the Console Launcher as well.

The junit-platform-launcher module also contains TestExecutionListener implementations that can be used for reporting purposes. See LoggingListener and SummaryGeneratingListener for details.

6.1.8. Configuring the Launcher

If you require fine-grained control over automatic detection and registration of test engines and test execution listeners, you may create an instance of LauncherConfig and supply that to the LauncherFactory.create(LauncherConfig) method. Typically an instance of LauncherConfig is created via the built-in fluent builder API, as demonstrated in the following example.

LauncherConfig launcherConfig = LauncherConfig.builder()
    .enableTestEngineAutoRegistration(false)
    .enableTestExecutionListenerAutoRegistration(false)
    .addTestEngines(new CustomTestEngine())
    .addTestExecutionListeners(new LegacyXmlReportGeneratingListener(reportsDir, out))
    .addTestExecutionListeners(new CustomTestExecutionListener())
    .build();

Launcher launcher = LauncherFactory.create(launcherConfig);

LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
    .selectors(selectPackage("com.example.mytests"))
    .build();

launcher.execute(request);