如何从 JUnit 内部运行 JMH 测试？

Question

如何使用JUnit测试在我现有的项目中运行JMH基准测试？官方留档建议制作一个单独的项目，使用Maven阴影插件，并在main方法中启动JMH。这是必要的吗？为什么推荐它？

Answer 1

@State(Scope.Benchmark)
@Threads(1)
@Fork(1)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@BenchmarkMode(Mode.All)
public class ToBytesTest {

  public static void main(String[] args) {
    ToBytesTest test = new ToBytesTest();
    System.out.println(test.string()[0] == test.charBufferWrap()[0] && test.charBufferWrap()[0] == test.charBufferAllocate()[0]);
  }

  @Test
  public void benchmark() throws Exception {
    org.openjdk.jmh.Main.main(new String[]{ToBytesTest.class.getName()});
  }

  char[] chars = new char[]{'中', '国'};

  @Benchmark
  public byte[] string() {
    return new String(chars).getBytes(StandardCharsets.UTF_8);
  }

  @Benchmark
  public byte[] charBufferWrap() {
    return StandardCharsets.UTF_8.encode(CharBuffer.wrap(chars)).array();
  }

  @Benchmark
  public byte[] charBufferAllocate() {
    CharBuffer cb = CharBuffer.allocate(chars.length).put(chars);
    cb.flip();
    return StandardCharsets.UTF_8.encode(cb).array();
  }
}

Answer 2

使用注释的声明性方法：

@State(Scope.Benchmark)
@Threads(1)
public class TestBenchmark {
       
    @Param({"10","100","1000"})
    public int iterations;


    @Setup(Level.Invocation)
    public void setupInvokation() throws Exception {
        // executed before each invocation of the benchmark
    }

    @Setup(Level.Iteration)
    public void setupIteration() throws Exception {
        // executed before each invocation of the iteration
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @Fork(warmups = 1, value = 1)
    @Warmup(batchSize = -1, iterations = 3, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(batchSize = -1, iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @OutputTimeUnit(TimeUnit.MILLISECONDS)
    public void test() throws Exception {
       Thread.sleep(ThreadLocalRandom.current().nextInt(0, iterations));
    }


    @Test
    public void benchmark() throws Exception {
        String[] argv = {};
        org.openjdk.jmh.Main.main(argv);
    }

}

Answer 3

我已经使用JUnit在我现有的Maven项目中运行了JMH，没有明显的不良影响。我无法回答为什么作者建议用不同的方式做事。我没有发现结果有什么不同。JMH推出了一个独立的JVM来运行基准测试，以隔离它们。我是这样做的:

>

将JMH依赖项添加到您的POM：

<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-core</artifactId>
  <version>1.21</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-generator-annprocess</artifactId>
  <version>1.21</version>
  <scope>test</scope>
</dependency>

请注意，我已经将它们放在< code>test范围内。

在Eclipse中，您可能需要手动配置注释处理器。NetBeans会自动处理此问题。

创建您的JUnit和JMH类。我选择将两者合并为一个类，但这取决于您。请注意，OptionsBuilder.include实际上决定了哪些基准将从您的JUnit测试中运行！

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.*;


public class TestBenchmark 
{

      @Test public void 
    launchBenchmark() throws Exception {

            Options opt = new OptionsBuilder()
                // Specify which benchmarks to run. 
                // You can be more specific if you'd like to run only one benchmark per test.
                .include(this.getClass().getName() + ".*")
                // Set the following options as needed
                .mode (Mode.AverageTime)
                .timeUnit(TimeUnit.MICROSECONDS)
                .warmupTime(TimeValue.seconds(1))
                .warmupIterations(2)
                .measurementTime(TimeValue.seconds(1))
                .measurementIterations(2)
                .threads(2)
                .forks(1)
                .shouldFailOnError(true)
                .shouldDoGC(true)
                //.jvmArgs("-XX:+UnlockDiagnosticVMOptions", "-XX:+PrintInlining")
                //.addProfiler(WinPerfAsmProfiler.class)
                .build();

            new Runner(opt).run();
        }

    // The JMH samples are the best documentation for how to use it
    // http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
    @State (Scope.Thread)
    public static class BenchmarkState
    {
        List<Integer> list;

          @Setup (Level.Trial) public void
        initialize() {

                Random rand = new Random();

                list = new ArrayList<>();
                for (int i = 0; i < 1000; i++)
                    list.add (rand.nextInt());
            }
    }

      @Benchmark public void 
    benchmark1 (BenchmarkState state, Blackhole bh) {

            List<Integer> list = state.list;

            for (int i = 0; i < 1000; i++)
                bh.consume (list.get (i));
        }
}

JMH 的注释处理器似乎不能很好地与 NetBeans 中的编译保存工作。每当修改基准测试时，您可能需要执行完整的清理和构建。（任何建议都值得赞赏！

运行您的启动基准测试并观察结果！

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.Foo
# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
# VM invoker: /usr/lib/jvm/java-8-jdk/jre/bin/java
# VM options: <none>
# Warmup: 2 iterations, 1 s each
# Measurement: 2 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 2 threads, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.Foo.benchmark1

# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration   1: 4.258 us/op
# Warmup Iteration   2: 4.359 us/op
Iteration   1: 4.121 us/op
Iteration   2: 4.029 us/op


Result "benchmark1":
  4.075 us/op


# Run complete. Total time: 00:00:06

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                                Mode  Cnt  Score   Error  Units
Foo.benchmark1                           avgt    2  4.075          us/op
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.013 sec

运行器甚至返回运行结果对象，您可以对其执行断言等。