05Proof
Confidence and provenance
Three different uncertainties hide behind the word confidence: was it seen, is it the same instant, and which state explains it. Provenance keeps them apart and keeps the answers out of the inputs.
- Uncertainty
- Provenance
- Leakage
- Observation confidence
- Was the ball or player really seen?
- Alignment confidence
- Do two sources show the same instant?
- Model confidence
- Which state best explains the evidence?
- Leakage rule
- Answer-bearing fields never become features
The design question
What does a possession system know about its own answer? “Confidence 0.8” means nothing until it is clear which doubt it measures. A design that reconciles witnesses or gates experts lives or dies on that distinction.
Three confidences
| Confidence | Question | Fails when |
|---|---|---|
| Observation | Was the object really measured on this frame? | An imputed value is stored as a measurement |
| Alignment | Do these two frames show the same instant? | A replay is joined as live |
| Model | Which state best explains the evidence? | Scores are uncalibrated, or trained on another provider |
The provenance record
Every value should be able to answer seven questions.
| Field | Answers |
|---|---|
| Source and native frame | Which witness produced it, and when on its own clock |
| Master-clock frame | Which tactical frame, or range of frames, it maps to |
| Observed or imputed | Was it measured, filled from another witness, or guessed |
| Tracking quality | How far the measurement itself can be trusted |
| Calibration quality | How far its position on the pitch can be trusted |
| Timing error | How far its instant can be trusted |
| Model and parameters | What produced the class scores |
This is the record that the single source of truth writes for every value it reconciles.
The leakage boundary
Some fields carry the answer. On open data such as the DFL release, the league’s possession and ball-status flags are the target a model is scored against. They can supervise training on other matches. They are never an input at inference, on any match. The same wall stands around any ground truth and any human note about the fixture being graded. The rules behind that wall are read as engineering requirements in The rules are the architecture.
What breaks it in real football
- False certainty in a scramble. The model reports 0.95 for home through a goalmouth melee because its inputs were interpolated, and nothing downstream knows.
- A confident wrong clock. Geometry is perfect and the frames are from different moments.
- A gate trusting the loudest expert. Without calibration, confidence measures temperament.
How to judge it
Draw it on held-out matches, then again on data from another provider. Then check that the frames flagged uncertain are the frames that are wrong more often.
Our read
One averaged confidence is a way to lose information. Carry the three separately, flag an ambiguous frame as ambiguous, and make every value able to say where it came from.