Java 11 — performance regressions against Java 8?

Question

UPDATE: Seeing as each method might be suffering from a different performance issue I decided to split this question into two:

1. Empty methods noticeably slower in Java 11 than Java 8
2. Consuming stack traces noticeably slower in Java 11 than Java 8

The original discussion can be found below...

---

I was comparing my library's performance under Java 8 and 11 when I ran across some surprising numbers. Here is the benchmark code:

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.infra.Blackhole;

import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
public class MyBenchmark
{
    @Benchmark
    public void emptyMethod()
    {
    }

    @Benchmark
    public void throwAndConsumeStacktrace(Blackhole bh)
    {
        try
        {
            throw new IllegalArgumentException("I love benchmarks");
        }
        catch (IllegalArgumentException e)
        {
            StringWriter sw = new StringWriter();
            e.printStackTrace(new PrintWriter(sw));
            bh.consume(sw.toString());
        }
    }
}

Running with jmh 1.21, OracleJDK 1.8.0_192 returns:

MyBenchmark.emptyMethod                avgt   25      0.363 ±   0.001  ns/op
MyBenchmark.throwAndConsumeStacktrace  avgt   25  21408.072 ± 127.393  ns/op

OracleJDK 11.0.1 returns:

Benchmark                              Mode  Cnt      Score      Error  Units
MyBenchmark.emptyMethod                avgt   25      0.759 ±    0.034  ns/op
MyBenchmark.throwAndConsumeStacktrace  avgt   25  47143.168 ± 1346.898  ns/op

OpenJDK 11.0.1 returns:

Benchmark                              Mode  Cnt      Score     Error  Units
MyBenchmark.emptyMethod                avgt   25      0.725 ±   0.001  ns/op
MyBenchmark.throwAndConsumeStacktrace  avgt   25  47389.051 ± 994.345  ns/op

Granted, the absolute difference for emptyMethod() is tiny but the trend seems to follow for more expensive operations like throwAndConsumeStacktrace(). As an aside, other operations (such as throwing exceptions and never consuming their stacktrace) are only moderately slower in Java 11 so the performance drop does not apply across all operations.

I understand that microbenchmarks do not indicate the performance behavior of real-life applications. Still, I'm curious where this difference is coming from. Any ideas?

Here is the full version info for the JDKs I used:

OracleJDK 8:

java version "1.8.0_192"
Java(TM) SE Runtime Environment (build 1.8.0_192-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode)

OracleJDK 11.0.1:

java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)

OpenJDK 11.0.1:

openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)