Understanding the score

The score in its pure form is just a number, and does not help us understand the make-up of the solution it represents. It doesn’t say which constraints are broken, or what caused them to break. To understand the score, it needs to be broken down.

The easiest way to do that during development is to print the score summary:

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
System.out.println(solutionManager.explain(vehicleRoutePlan));

For example, in vehicle routing, this prints that talk vehicle '1' matched the soft constraint minimizeTravelTime:

Explanation of score (0hard/-10343soft):
    Constraint matches:
        -10343soft: constraint (minimizeTravelTime) has 6 matches:
            -2335soft: justified with (MinimizeTravelTimeJustification[vehicleName=1, totalDrivingTimeSeconds=2335, description=Vehicle '1' total travel time is 0 hours 39 minutes.])
            ...
        ...
    Indictments (top 5 of 6):
        -2335soft: indicted with (1) has 1 matches:
            -2335soft: constraint (minimizeTravelTime)
        ...

Do not attempt to parse this string or expose it in services. It serves for debugging purposes only. Use score analysis instead.

In the string above, there are some previously unexplained concepts.

  • A Constraint match is created every time a constraint causes a change to the score.

  • Justifications are user-defined objects that implement the ai.timefold.solver.core.api.score.stream.ConstraintJustification interface, which carry meaningful information about a constraint match, such as its package, name and any metadata that the user chooses to expose. Justifications are most easily available via score analysis.

  • Indicted objects are objects which were directly involved in causing a constraint to match. For example, if your constraints penalize each vehicle, then there will be one ai.timefold.solver.core.api.score.constraint.Indictment instance per vehicle, carrying the vehicle as an indicted object. Indictments are typically used for heat map visualization.

Constraint Streams API can analyze the score natively. Incremental Java score calculation requires implementing an extra interface. Easy Java score calculation does not support score explanation.

1. Score Analysis: Which constraints are broken?

If other parts of your application, such as your web interface, need to describe the solution to the user, use the SolutionManager API:

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
ScoreAnalysis<HardSoftLongScore> scoreAnalysis = solutionManager.analyze(vehicleRoutePlan);

ScoreAnalysis is a JSON-friendly representation of the score, breaking down the score into individual constraints. Using score analysis, you can find out:

  • What’s the total score.

  • Which constraints are broken, and how many times.

  • Which planning entities and problem facts are responsible for breaking which constraints.

1.1. Finding the broken constraints

When you have the ScoreAnalysis instance, you can find out which constraints are broken:

scoreAnalysis.constraintMap().forEach((constraintRef, constraintAnalysis) -> {
   String constraintId = constraintRef.constraintId();
   HardSoftScore scorePerConstraint = constraintAnalysis.score();
   ...
});

If you wish to go further and find out which planning entities and problem facts are responsible for breaking the constraint, you can further explore the ConstraintAnalysis instance you received from ScoreAnalysis.constraintMap():

int matchCount = constraintAnalysis.matchCount();
constraintAnalysis.matches().forEach(matchAnalysis -> {
    HardSoftScore scorePerMatch = matchAnalysis.score();
    Object justification = matchAnalysis.justification();
    ...
});

Each match is accompanied by the score difference it caused, and a justification object (see above). Typically, the scoring engine creates justification objects automatically by using the results of Constraint Streams' justifyWith(…​) call.

1.2. Identifying changes between two solutions

If you have two different solutions from the Solver, you can compare them using ScoreAnalysis and find out what changed between them:

ScoreAnalysis<HardSoftScore>  firstAnalysis = solutionManager.analyze(firstSolution);
ScoreAnalysis<HardSoftScore> secondAnalysis = solutionManager.analyze(secondSolution);
ScoreAnalysis<HardSoftScore>           diff = firstAnalysis.diff(secondAnalysis);

// Score difference only carries the constraints whose matches changed from first to second solution.
diff.constraintMap().forEach((constraintRef, constraintAnalysis) -> {
   String constraintId = constraintRef.constraintId();
   HardSoftScore scoreDiff = constraintAnalysis.score();
   // Matches only include constraint matches that:
   //   - the second solution either added to or removed from the first solution.
   //   - had their score changed.
   // Two matches are considered equal if their justification objects are equal.
   constraintAnalysis.matches().forEach(matchAnalysis -> {
       ...
   });
});

Think of diff(…​) as a subtraction operation, where the second solution is subtracted from the first solution. For example, if the first solution has score of 2hard/3soft and the second solution has score of 1hard/2soft, then the score difference will be 1hard/1soft, indicating that the second solution is better than the first solution.

The same applies to constraints and constraint matches. If a constraint did not match in the first solution but did match in the second, then the constraint match will be included in the diff as negative. If instead the constraint did match in the first solution but did not match in the second, then the constraint match will be included in the diff as positive.

1.3. Sending score analysis over the wire

The purpose of ScoreAnalysis is to break down the score so that the end user can understand it. To succeed at this, ScoreAnalysis is JSON-friendly and can be easily sent over the wire from the backend to the frontend.

ScoreAnalysis instances will serialize into JSON automatically (using Jackson):

If you implemented ConstraintJustication to provide custom justification objects, you are responsible for making them JSON-friendly yourself.

ScoreAnalysis doesn’t natively deserialize from JSON back to Java objects. This is because we have no way of knowing which Score or ConstraintJustification implementations you may be using. However, deserialization is easy to implement yourself by extending AbstractScoreAnalysisJacksonDeserializer and registering it with Jackson’s ObjectMapper.

2. Heat map: Visualize the hot planning entities

To show a heat map in the UI that highlights the planning entities and problem facts have an impact on the Score, get the Indictment map from the ScoreExplanation:

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
ScoreExplanation<VehicleRoutePlan, HardSoftLongScore> scoreExplanation = solutionManager.explain(vehicleRoutePlan);
Map<Object, Indictment<HardSoftLongScore>> indictmentMap = scoreExplanation.getIndictmentMap();
for (Visit visit : vehicleRoutePlan.getVisits()) {
    Indictment<HardSoftLongScore> indictment = indictmentMap.get(visit);
    if (indictment == null) {
        continue;
    }
    // The score impact of that planning entity
    HardSoftLongScore totalScore = indictment.getScore();

    for (ConstraintMatch<HardSoftLongScore> constraintMatch : indictment.getConstraintMatchSet()) {
        String constraintName = constraintMatch.getConstraintName();
        HardSoftLongScore score = constraintMatch.getScore();
        ...
    }
}

ScoreExplanation should only be used for processing indictments. For analyzing the score and processing constraint matches, use score analysis instead, which is faster and JSON-friendly.

Each Indictment is the sum of all constraints where that justification object is involved with. The sum of all the Indictment.getScoreTotal() differs from the overall score, because multiple Indictments can share the same ConstraintMatch.

scoreVisualization