1.2.7.2. JUnit Platform Test Kit

优质

小牛编辑

131浏览

2023-12-01

The junit-platform-testkit artifact provides support for executing a test plan on the JUnit Platform and then verifying the expected results. As of JUnit Platform 1.4, this support is limited to the execution of a single TestEngine (see Engine Test Kit).

Although the Test Kit is currently an experimental feature, the JUnit Team invites you to try it out and provide feedback to help improve the Test Kit APIs and eventually promote this feature.

6.2.1. Engine Test Kit

The org.junit.platform.testkit.engine package provides support for executing a TestPlan for a given TestEngine running on the JUnit Platform and then accessing the results via a fluent API to verify the expected results. The key entry point into this API is the EngineTestKit which provides static factory methods named engine() and execute(). It is recommended that you select one of the engine() variants to benefit from the fluent API for building a LauncherDiscoveryRequest.

If you prefer to use the LauncherDiscoveryRequestBuilder from the Launcher API to build your LauncherDiscoveryRequest, you must use one of the execute() variants in EngineTestKit.

The following test class written using JUnit Jupiter will be used in subsequent examples.

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assumptions.assumeTrue;

import example.util.Calculator;

import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(OrderAnnotation.class)
public class ExampleTestCase {

    private final Calculator calculator = new Calculator();

    @Test
    @Disabled("for demonstration purposes")
    @Order(1)
    void skippedTest() {
        // skipped ...
    }

    @Test
    @Order(2)
    void succeedingTest() {
        assertEquals(42, calculator.multiply(6, 7));
    }

    @Test
    @Order(3)
    void abortedTest() {
        assumeTrue("abc".contains("Z"), "abc does not contain Z");
        // aborted ...
    }

    @Test
    @Order(4)
    void failingTest() {
        // The following throws an ArithmeticException: "/ by zero"
        calculator.divide(1, 0);
    }

}

For the sake of brevity, the following sections demonstrate how to test JUnit’s own JupiterTestEngine whose unique engine ID is "junit-jupiter". If you want to test your own TestEngine implementation, you need to use its unique engine ID. Alternatively, you may test your own TestEngine by supplying an instance of it to the EngineTestKit.engine(TestEngine) static factory method.

6.2.2. Asserting Statistics

One of the most common features of the Test Kit is the ability to assert statistics against events fired during the execution of a TestPlan. The following tests demonstrate how to assert statistics for containers and tests in the JUnit Jupiter TestEngine. For details on what statistics are available, consult the Javadoc for EventStatistics.

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;

import example.ExampleTestCase;

import org.junit.jupiter.api.Test;
import org.junit.platform.testkit.engine.EngineTestKit;

class EngineTestKitStatisticsDemo {

    @Test
    void verifyJupiterContainerStats() {
        EngineTestKit
            .engine("junit-jupiter") 
            .selectors(selectClass(ExampleTestCase.class)) 
            .execute() 
            .containerEvents() 
            .assertStatistics(stats -> stats.started(2).succeeded(2)); 
    }

    @Test
    void verifyJupiterTestStats() {
        EngineTestKit
            .engine("junit-jupiter") 
            .selectors(selectClass(ExampleTestCase.class)) 
            .execute() 
            .testEvents() 
            .assertStatistics(stats ->
                stats.skipped(1).started(3).succeeded(1).aborted(1).failed(1)); 
    }

}

	Select the JUnit Jupiter `TestEngine`.
	Select the `ExampleTestCase` test class.
	Execute the `TestPlan`.
	Filter by container events.
	Assert statistics for container events.
	Filter by test events.
	Assert statistics for test events.

In the verifyJupiterContainerStats() test method, the counts for the started and succeeded statistics are 2 since the JupiterTestEngine and the ExampleTestCase class are both considered containers.

6.2.3. Asserting Events

If you find that asserting statistics alone is insufficient for verifying the expected behavior of test execution, you can work directly with the recorded Event elements and perform assertions against them.

For example, if you want to verify the reason that the skippedTest() method in ExampleTestCase was skipped, you can do that as follows.

The assertThatEvents() method in the following example is a shortcut for org.assertj.core.api.Assertions.assertThat(events.list()) from the AssertJ assertion library.

For details on what conditions are available for use with AssertJ assertions against events, consult the Javadoc for EventConditions.

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectMethod;
import static org.junit.platform.testkit.engine.EventConditions.event;
import static org.junit.platform.testkit.engine.EventConditions.skippedWithReason;
import static org.junit.platform.testkit.engine.EventConditions.test;

import example.ExampleTestCase;

import org.junit.jupiter.api.Test;
import org.junit.platform.testkit.engine.EngineTestKit;
import org.junit.platform.testkit.engine.Events;

class EngineTestKitSkippedMethodDemo {

    @Test
    void verifyJupiterMethodWasSkipped() {
        String methodName = "skippedTest";

        Events testEvents = EngineTestKit 
            .engine("junit-jupiter") 
            .selectors(selectMethod(ExampleTestCase.class, methodName)) 
            .execute() 
            .testEvents(); 

        testEvents.assertStatistics(stats -> stats.skipped(1)); 

        testEvents.assertThatEvents() 
            .haveExactly(1, event(test(methodName),
                skippedWithReason("for demonstration purposes")));
    }

}

	Select the JUnit Jupiter `TestEngine`.
	Select the `skippedTest()` method in the `ExampleTestCase` test class.
	Execute the `TestPlan`.
	Filter by test events.
	Save the test `Events` to a local variable.
	Optionally assert the expected statistics.
	Assert that the recorded test events contain exactly one skipped test named `skippedTest` with `"for demonstration purposes"` as the reason.

If you want to verify the type of exception thrown from the failingTest() method in ExampleTestCase, you can do that as follows.

For details on what conditions are available for use with AssertJ assertions against events and execution results, consult the Javadoc for EventConditions and TestExecutionResultConditions, respectively.

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
import static org.junit.platform.testkit.engine.EventConditions.event;
import static org.junit.platform.testkit.engine.EventConditions.finishedWithFailure;
import static org.junit.platform.testkit.engine.EventConditions.test;
import static org.junit.platform.testkit.engine.TestExecutionResultConditions.instanceOf;
import static org.junit.platform.testkit.engine.TestExecutionResultConditions.message;

import example.ExampleTestCase;

import org.junit.jupiter.api.Test;
import org.junit.platform.testkit.engine.EngineTestKit;

class EngineTestKitFailedMethodDemo {

    @Test
    void verifyJupiterMethodFailed() {
        EngineTestKit.engine("junit-jupiter") 
            .selectors(selectClass(ExampleTestCase.class)) 
            .execute() 
            .testEvents() 
            .assertThatEvents().haveExactly(1, 
                event(test("failingTest"),
                    finishedWithFailure(
                        instanceOf(ArithmeticException.class), message("/ by zero"))));
    }

}

	Select the JUnit Jupiter `TestEngine`.
	Select the `ExampleTestCase` test class.
	Execute the `TestPlan`.
	Filter by test events.
	Assert that the recorded test events contain exactly one failing test named `failingTest` with an exception of type `ArithmeticException` and an error message equal to `"/ by zero"`.

Although typically unnecessary, there are times when you need to verify all of the events fired during the execution of a TestPlan. The following test demonstrates how to achieve this via the assertEventsMatchExactly() method in the EngineTestKit API.

Since assertEventsMatchExactly() matches conditions exactly in the order in which the events were fired, ExampleTestCase has been annotated with @TestMethodOrder(OrderAnnotation.class) and each test method has been annotated with @Order(…). This allows us to enforce the order in which the test methods are executed, which in turn allows our verifyAllJupiterEvents() test to be reliable.

If you want to do a partial match with or without ordering requirements, you can use the methods assertEventsMatchLooselyInOrder() and assertEventsMatchLoosely(), respectively.

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
import static org.junit.platform.testkit.engine.EventConditions.abortedWithReason;
import static org.junit.platform.testkit.engine.EventConditions.container;
import static org.junit.platform.testkit.engine.EventConditions.engine;
import static org.junit.platform.testkit.engine.EventConditions.event;
import static org.junit.platform.testkit.engine.EventConditions.finishedSuccessfully;
import static org.junit.platform.testkit.engine.EventConditions.finishedWithFailure;
import static org.junit.platform.testkit.engine.EventConditions.skippedWithReason;
import static org.junit.platform.testkit.engine.EventConditions.started;
import static org.junit.platform.testkit.engine.EventConditions.test;
import static org.junit.platform.testkit.engine.TestExecutionResultConditions.instanceOf;
import static org.junit.platform.testkit.engine.TestExecutionResultConditions.message;

import java.io.StringWriter;
import java.io.Writer;

import example.ExampleTestCase;

import org.junit.jupiter.api.Test;
import org.junit.platform.testkit.engine.EngineTestKit;
import org.opentest4j.TestAbortedException;

class EngineTestKitAllEventsDemo {

    @Test
    void verifyAllJupiterEvents() {
        Writer writer = // create a java.io.Writer for debug output

        EngineTestKit.engine("junit-jupiter") 
            .selectors(selectClass(ExampleTestCase.class)) 
            .execute() 
            .allEvents() 
            .debug(writer) 
            .assertEventsMatchExactly( 
                event(engine(), started()),
                event(container(ExampleTestCase.class), started()),
                event(test("skippedTest"), skippedWithReason("for demonstration purposes")),
                event(test("succeedingTest"), started()),
                event(test("succeedingTest"), finishedSuccessfully()),
                event(test("abortedTest"), started()),
                event(test("abortedTest"),
                    abortedWithReason(instanceOf(TestAbortedException.class),
                        message(m -> m.contains("abc does not contain Z")))),
                event(test("failingTest"), started()),
                event(test("failingTest"), finishedWithFailure(
                    instanceOf(ArithmeticException.class), message("/ by zero"))),
                event(container(ExampleTestCase.class), finishedSuccessfully()),
                event(engine(), finishedSuccessfully()));
    }

}

	Select the JUnit Jupiter `TestEngine`.
	Select the `ExampleTestCase` test class.
	Execute the `TestPlan`.
	Filter by all events.
	Print all events to the supplied `writer` for debugging purposes. Debug information can also be written to an `OutputStream` such as `System.out` or `System.err`.
	Assert all events in exactly the order in which they were fired by the test engine.

The debug() invocation from the preceding example results in output similar to the following.

All Events:
    Event [type = STARTED, testDescriptor = JupiterEngineDescriptor: [engine:junit-jupiter], timestamp = 2018-12-14T12:45:14.082280Z, payload = null]
    Event [type = STARTED, testDescriptor = ClassTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase], timestamp = 2018-12-14T12:45:14.089339Z, payload = null]
    Event [type = SKIPPED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:skippedTest()], timestamp = 2018-12-14T12:45:14.094314Z, payload = 'for demonstration purposes']
    Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:succeedingTest()], timestamp = 2018-12-14T12:45:14.095182Z, payload = null]
    Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:succeedingTest()], timestamp = 2018-12-14T12:45:14.104922Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]]
    Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:abortedTest()], timestamp = 2018-12-14T12:45:14.106121Z, payload = null]
    Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:abortedTest()], timestamp = 2018-12-14T12:45:14.109956Z, payload = TestExecutionResult [status = ABORTED, throwable = org.opentest4j.TestAbortedException: Assumption failed: abc does not contain Z]]
    Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:failingTest()], timestamp = 2018-12-14T12:45:14.110680Z, payload = null]
    Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:failingTest()], timestamp = 2018-12-14T12:45:14.111217Z, payload = TestExecutionResult [status = FAILED, throwable = java.lang.ArithmeticException: / by zero]]
    Event [type = FINISHED, testDescriptor = ClassTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase], timestamp = 2018-12-14T12:45:14.113731Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]]
    Event [type = FINISHED, testDescriptor = JupiterEngineDescriptor: [engine:junit-jupiter], timestamp = 2018-12-14T12:45:14.113806Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]]