Java 22 is a new release of the Java platform, the trusty programming language that Timefold Solver is written in. It brings a bunch of new features, as well as the usual bugfixes and smaller improvements.

Java 22 will be generally available on March 19, 2024, but you can already try it out using the release candidate builds. We find that the easiest way to get started with Java 22 is to use SDKMAN, and that is what we did as well. We have also decided to test GraalVM for JDK 22. You may know GraalVM as the tool to run Java natively, but today we actually mean the regular JDK distribution that GraalVM provides.

While long-term support won’t likely be available for Java 22, it is still a good idea to try it out just to make sure your code is still in good shape and will be ready for the next LTS. For Timefold Solver, that means making sure the entire codebase continues to work flawlessly on Java 22, as well as running some benchmarks to ensure our users can expect at least the same performance as before. Let’s get started on that.

We’ll start with score director micro-benchmarks, which we use regularly to establish the impact of various changes on the performance of Constraint Streams. These benchmarks do not run the entire solver; rather, they focus exclusively on the score calculation part of the solver.

They are implemented using Java Microbenchmark Harness (JMH), and they run in many Java Virtual Machine (JVM) forks and with sufficient warmup. This gives us a good level of confidence in the results. In fact, the margin of error on these numbers is only ± 2%.

Here is the Constraint Streams performance on Java 22 vs. Java 21:

In all cases but one, the difference between OpenJDK 22 and 21 is within the margin of error for the benchmark. What that means is that we have observed no statistifically significant change in performance between the two releases. The "Tennis" benchmark is the only outlier, and it will be interesting for us to look into it and find out why.

The more interesting finding is that GraalVM for JDK 22 can be significantly faster than OpenJDK 22! With an average performance improvement of ~5% across the benchmarks, and a maximum improvement of ~15% in the "Conference Scheduling" benchmark, GraalVM is definitely worth considering for your Timefold Solver application.

It should be noted that we ran these benchmarks with ParallelGC as the garbage collector (GC), instead of the default G1GC. We are making this choice because ParallelGC has been the best GC for the solver in the past, as the solver is all about throughput and not latency.

Now that we’ve seen the micro-benchmarks, it’s time to compare them to real-world solver performance. This includes the entire solver, not just the score calculation part.

We ran the solver manually in 10 different JVM forks, and used the median score calculation speed. We selected a subset of the available benchmarks, to keep the run time short; the selection is representative of the entire benchmark suite in terms of heuristics used and code paths exercised. Once again, ParallelGC is used as the garbage collector. Here are the results:

The numbers confirm the results of the micro-benchmarks. The difference between OpenJDK 22 and OpenJDK 21 is negligible, while GraalVM for JDK 22 brings improvements of up to 10 %.

Since we haven’t established a formal interval of confidence for these large benchmarks, we can’t say with certainty that the improvements are statistically significant. However, the fluctuations observed between runs have been small enough to give us confidence in the results.

In this post, we’ve shown that:

Even though OpenJDK 22 doesn’t bring any performance improvements for us, we still encourage you to try Java 22. It is free after all, and you’ll be able to enjoy the latest and greatest Java platform.