OptaPlanner EOL notice - Timefold offers support
By

As the Java 22 release is fast approaching, we took our usual look at its performance when it comes to the Timefold Solver. Read on to find out how the solver performs on Java 22, compared to Java 21. This time including GraalVM!

But first, let’s get some things out of the way.

What is Java 22 and how to get it

Java 22 is a new release of the Java platform, the trusty programming language that Timefold Solver is written in. It brings a bunch of new features, as well as the usual bugfixes and smaller improvements.

Java 22 will be generally available on March 19, 2024, but you can already try it out using the release candidate builds. We find that the easiest way to get started with Java 22 is to use SDKMAN, and that is what we did as well. We have also decided to test GraalVM for JDK 22. You may know GraalVM as the tool to run Java natively, but today we actually mean the regular JDK distribution that GraalVM provides.

While long-term support won’t likely be available for Java 22, it is still a good idea to try it out just to make sure your code is still in good shape and will be ready for the next LTS. For Timefold Solver, that means making sure the entire codebase continues to work flawlessly on Java 22, as well as running some benchmarks to ensure our users can expect at least the same performance as before. Let’s get started on that.

Micro-benchmarks

We’ll start with score director micro-benchmarks, which we use regularly to establish the impact of various changes on the performance of Constraint Streams. These benchmarks do not run the entire solver; rather, they focus exclusively on the score calculation part of the solver.

They are implemented using Java Microbenchmark Harness (JMH), and they run in many Java Virtual Machine (JVM) forks and with sufficient warmup. This gives us a good level of confidence in the results. In fact, the margin of error on these numbers is only ± 2%.

Here is the Constraint Streams performance on Java 22 vs. Java 21:

Performance of OpenJDK 22 is unchanged relatively to 21

In all cases but one, the difference between OpenJDK 22 and 21 is within the margin of error for the benchmark. What that means is that we have observed no statistifically significant change in performance between the two releases. The "Tennis" benchmark is the only outlier, and it will be interesting for us to look into it and find out why.

The more interesting finding is that GraalVM for JDK 22 can be significantly faster than OpenJDK 22! With an average performance improvement of ~5% across the benchmarks, and a maximum improvement of ~15% in the "Conference Scheduling" benchmark, GraalVM is definitely worth considering for your Timefold Solver application.

It should be noted that we ran these benchmarks with ParallelGC as the garbage collector (GC), instead of the default G1GC. We are making this choice because ParallelGC has been the best GC for the solver in the past, as the solver is all about throughput and not latency.

Real-world benchmarks

Now that we’ve seen the micro-benchmarks, it’s time to compare them to real-world solver performance. This includes the entire solver, not just the score calculation part.

We ran the solver manually in 10 different JVM forks, and used the median score calculation speed. We selected a subset of the available benchmarks, to keep the run time short; the selection is representative of the entire benchmark suite in terms of heuristics used and code paths exercised. Once again, ParallelGC is used as the garbage collector. Here are the results:

OpenJDK performance unchanged

The numbers confirm the results of the micro-benchmarks. The difference between OpenJDK 22 and OpenJDK 21 is negligible, while GraalVM for JDK 22 brings improvements of up to 10 %.

Since we haven’t established a formal interval of confidence for these large benchmarks, we can’t say with certainty that the improvements are statistically significant. However, the fluctuations observed between runs have been small enough to give us confidence in the results.

Conclusion

In this post, we’ve shown that:

  • Timefold Solver 1.8.0 works perfectly fine with Java 22, no changes are needed. Switching to the latest version of Java is a non-event for us here at Timefold, a testament to Java’s backward compatibility promise.

  • Switching to Java 22 likely won’t bring any performance improvements or regressions for the solver.

  • GraalVM for JDK 22 can bring performance improvements of up to 10 % for the solver.

Even though OpenJDK 22 doesn’t bring any performance improvements for us, we still encourage you to try Java 22. It is free after all, and you’ll be able to enjoy the latest and greatest Java platform.

Appendix A: Reproducing the results

These benchmarks use Timefold Solver 1.8.0, the latest version at the time of writing.

All benchmarks were run on Fedora Linux 39, with Intel Core i7-12700H CPU and 32 GB of RAM. We used the following JDK distributions:

  • OpenJDK Runtime Environment Temurin-21.0.2+13 (21.0.2+13-LTS), available as 21.0.2-tem on SDKMAN.

  • OpenJDK Runtime Environment 22.ea.36 (22+36-2370), available as 22.ea.36-open on SDKMAN.

  • GraalVM for JDK 22.ea.06 (22-dev+36.1), available on Github.

Source code of the micro-benchmarks can be found in my own personal repository. The real-world benchmarks were run using timefold-solver-benchmark, using this configuration. (You’ll need timefold-solver-examples on the classpath.) Raw data used for the charts can be found in this spreadsheet.

Continue reading

  • Red Hat: OptaPlanner End Of Life Notice (EOL)

    Timefold, led by former core OptaPlanner engineers, offers a seamless transition with extended support and accelerated innovation.

  • Newsletter 4: A big Speed Upgrade for Timefold - Our First Customer Story and AMA!

    Unlock unprecedented speed with Timefold's latest update! Essential for OptaPlanner and pre-1.8.0 users – upgrade now for instant gains

  • Continuous Planning Optimization with Pinning

    Discover how to make non-disruptive, feasible adjustments to your already in-progress plans with Timefold, ensuring real-time adaptability to unexpected changes.

  • Fast Planning Optimization with the Recommended Fit API

    Discover how Timefold's Recommended Fit API offers swift, feasible adjustments to your plans, ensuring real-time adaptability to unexpected changes.

  • Newsletter 3: Explainable AI for Planning Optimization

    Discover how Timefold's ScoreAnalysis API builds trust in automated planning. And don't forget to join Timefold's first AMA on Youtube Live on March 19th!

  • Use Multiple Solvers in the Same Application

    Learn how to configure and inject multiple SolverManager instances in one application using Timefold Solver, tackling distinct planning problems seamlessly.

Sign up for our newsletter

And stay up to date with announcements, the latest news, events, roadmap progress & product updates from Timefold!

We care about the protection of your data. Read our Privacy Policy.