How the forecast works
Before every Grand Prix, this site publishes how it thinks the race will finish. Here is how that forecast is made, and how often it is right.
What it looks at
- Where each driver starts
- Their recent results
- How fast the car has been
- One-lap speed
- How often the car breaks down
- Their record at this track
- This weekend's practice times (for qualifying)
- Sprint qualifying, on sprint weekends (for qualifying)
From every race since 2018 it learns how much each of these tends to matter, then ranks this weekend's drivers the same way. It never sees a race's result before forecasting it.
From a ranking to a percentage
A ranking is not a forecast yet. So the race is played out 10,000 times in two ways: by drawing whole finishing orders from the ranking itself, and in a simple race simulation that adds retirements, safety cars and the pull of the starting grid. The two are blended, in a proportion set on past seasons.
If a driver comes first in 36% of the blended results, their chance to win is 36%: about 36 in every 100. Win, podium, top 5 and top 10 are all read from the same blended finishing orders, so they never contradict each other.
It sharpens through the weekend
A new forecast is published at each step of a race weekend, and none is edited afterwards. Each one knows a little more than the last.
- Before practicePast races only, on the Wednesday of race week.
- After practiceAdds this weekend's lap times.
- After qualifyingAdds the real starting grid.
- After the raceGraded against the result.
At the Singapore Grand Prix, the forecast gave Verstappen 18% before practice, and 41% once the grid was set. Verstappen won.
Is it any good?
To find out, it forecast every race from 2024 onward using only the races before each one, and was compared with simple guesses anyone could make.
- After qualifying, its favourite won 68% of races. That sounds good, but simply backing the driver on pole won 60%, and over 65 races a gap that size could be luck. The grid itself holds most of what can be predicted.
- Before qualifying, with or without this weekend's practice, it beats following the championship table. Its chances are more accurate, though it picks the winner no more often.
- When it says 30%, it happens about 30% of the time. Its boldest calls are the exception: podiums it rated around 94% happened 79% of the time.
- Its title favourite went on to win 89% of the time. But that favourite was nearly always the driver already leading the championship (26 times in 27), and simply backing the leader would have been right 85% of the time.
Does 30% mean 30%?
Every driver in every race since 2024, grouped by the chance the forecast gave. On the dashed line, the forecast meant exactly what it said. Each whisker is the 95% interval on how often it happened. Bands with fewer than 10 cases are left out.
What it can’t see
Weather, tyre strategy, a safety car at the wrong moment, penalties handed out during the race, upgrades and team orders. It also treats each car's chance of breaking down separately, even when a team shares a problem.
All the numbers
For anyone who wants the detail: every comparison the evaluation made, model minus reference, with its 95% interval. A difference only counts when its interval excludes zero. Log loss and Brier score measure how far the chances were from what happened, and NDCG how well the top places were ordered. ↓ marks a score where lower is better.
| Measure | Model | Reference | Difference | Verdict |
|---|---|---|---|---|
| After qualifying vs the calibrated starting grid | ||||
| Top-3 orderNDCG@3 | 0.856 | 0.861 | -0.005 (-0.030 to +0.020) | no clear difference |
| Top-5 orderNDCG@5 | 0.897 | 0.900 | -0.003 (-0.019 to +0.014) | no clear difference |
| Winner calledhit rate | 68% | 60% | +8 pts (-3 to +18) | no clear difference |
| Podium namedoverlap of 3 | 1.938 | 2.062 | -0.123 (-0.262 to +0.015) | no clear difference |
| Top five namedoverlap of 5 | 3.892 | 3.908 | -0.015 (-0.169 to +0.139) | no clear difference |
| Whole-field orderSpearman | 0.656 | 0.679 | -0.023 (-0.048 to +0.001) | no clear difference |
| Whole-field orderKendall | 0.522 | 0.549 | -0.027 (-0.052 to -0.002) | reference clearly better |
| Win chanceslog loss ↓ | 1.173 | 1.309 | -0.136 (-0.349 to +0.065) | no clear difference |
| Win chancesBrier ↓ | 0.560 | 0.589 | -0.029 (-0.118 to +0.065) | no clear difference |
| Podium chancesBrier ↓ | 0.067 | 0.068 | -0.000 (-0.006 to +0.005) | no clear difference |
| Top-10 chancesBrier ↓ | 0.149 | 0.154 | -0.005 (-0.014 to +0.004) | no clear difference |
| Before qualifying vs the championship order | ||||
| Top-3 orderNDCG@3 | 0.766 | 0.751 | +0.015 (-0.025 to +0.056) | no clear difference |
| Top-5 orderNDCG@5 | 0.837 | 0.803 | +0.034 (+0.003 to +0.067) | model clearly better |
| Winner calledhit rate | 32% | 29% | +3 pts (-8 to +14) | no clear difference |
| Podium namedoverlap of 3 | 1.754 | 1.677 | +0.077 (-0.092 to +0.246) | no clear difference |
| Top five namedoverlap of 5 | 3.662 | 3.431 | +0.231 (+0.031 to +0.462) | model clearly better |
| Whole-field orderSpearman | 0.579 | 0.573 | +0.007 (-0.033 to +0.053) | no clear difference |
| Whole-field orderKendall | 0.441 | 0.439 | +0.002 (-0.032 to +0.041) | no clear difference |
| Win chanceslog loss ↓ | 1.755 | 2.169 | -0.414 (-0.667 to -0.154) | model clearly better |
| Win chancesBrier ↓ | 0.796 | 0.867 | -0.071 (-0.152 to +0.011) | no clear difference |
| Podium chancesBrier ↓ | 0.085 | 0.095 | -0.010 (-0.017 to -0.002) | model clearly better |
| Top-10 chancesBrier ↓ | 0.174 | 0.176 | -0.001 (-0.011 to +0.008) | no clear difference |
| Qualifying vs recent qualifying form | ||||
| Pole-sitter calledhit rate | 34% | 26% | +8 pts (-6 to +22) | no clear difference |
| Pole chanceslog loss ↓ | 1.681 | 2.105 | -0.424 (-0.653 to -0.194) | model clearly better |
| Top-5 orderNDCG@5 | 0.857 | 0.810 | +0.046 (+0.025 to +0.069) | model clearly better |
| Top ten namedoverlap of 10 | 8.000 | 7.785 | +0.215 (+0.000 to +0.446) | no clear difference |
| Whole-field orderSpearman | 0.733 | 0.716 | +0.018 (-0.005 to +0.044) | no clear difference |
Source: reports/backtest.json, written by make backtest.
Under the hood
- The model
- Ranks the drivers within each raceXGBoost, learning to rank
- The chances
- Turns the ranking into odds, blended with 10,000 simulated racesPlackett–Luce, Monte Carlo
- The test
- Settings picked on 2022–2023, then every result measured on races from 2024 onward that it had never seenwalk-forward evaluation
- The comparison
- Only counts a gap that survives reshuffling the racespaired bootstrap