02Rules and proof

The rules are the architecture

No hand labels. No tuning on the match. No match data in the cloud. An exact file anyone can regenerate. Read as engineering requirements, the rules design most of the system, and they decide what counts as proof.

  • Local only
  • No hand labels
  • Proof on external labels
Match data crosses one boundary only: into an offline container on one laptop. What comes out is a label file and aggregate metrics.
Labels on the match
None, by anyone, for any purpose
Public answers
Correctness check only
Match data
Never leaves the machine
Result
Algorithmic and exactly reproducible

Four rules, four design consequences

The rule What it removes What it produces
No human labels or corrections on the match, even a small validation set Supervised learning on the fixture Fixed football evidence and an exact decoder
The public five-minute answers verify a pipeline; they never train or tune it Model selection on the match Selection on external labels
No match data, and no extracts of it, to hosted or cloud AI services Cloud GPUs, hosted models on the data Everything computed locally
The scored file must be algorithmic and exactly reproducible Manual fixes, hidden state, randomness A deterministic, pinned pipeline

Three more close the remaining doors. Watching the video to understand the data is fine; turning what is seen into labels or frame-specific rules is not. Outside information about this match, including third-party event or possession data, is prohibited. And the restricted data, or anything that could reconstruct it, never enters a public repository.

What stays open

The rules are precise about what they allow, and it is a lot.

Material Allowed use
Other matches, openly labelled or annotated by hand Train, tune, select, report
The same matches, degraded Stress a method under the target’s conditions
Simulated matches with known answers Explore, search, unit-test behaviour
Downloaded model weights Run on the participant’s own machine, where nothing is transmitted
Published baseline scores Reference points
The public five-minute answers Check frame alignment and label semantics, nothing else

Model eligibility and data transmission are separate questions. A provider’s zero-retention promise is not a permission: what matters is whether the data leaves the machine, not what happens to it afterwards.

What counts as proof

With no label on the match, a result has to be proved somewhere else. Claims in football AI come in four strengths, and mixing them is how weak systems get strong reputations.

Level What it is What it proves
Official fact The organisers’ documents and scorer What the task and the metric are
Published result A paper’s numbers on its own dataset The method works there
Running code A public repository that reproduces The method exists and executes
Hypothesis An idea with no independent test Nothing yet

Every number on this site sits at a stated level. Organisers’ figures are cited to their documents. Our own were produced by the official, unchanged scorer on open data anyone can download.

Our proof is a ladder with one job per rung: simulated matches to explore, a real match with external labels to decide, the public window to check the file. How it ran is on the Pareto lens. Why it is built that way is in Two metrics, one verdict. The numbers are on the Initial Challenge page, and the data behind them is DFL IDSSE open data.

Where each rule lands in our build

Rule In the build Read more
No labels Fixed football evidence, an exact decode, seven parameters set away from the match The Shrewdies decoder
Public window Checked format, frame alignment and team orientation. Chose nothing The Pareto lens
Local only An offline container on one laptop, read-only inputs, one writable output folder Security and data protection
Reproducible Pinned image and dependencies, no randomness, the organisers’ validator on every file Reproducibility

Nothing from the match is published. The repository holds no match data, and this site shows aggregate metrics, synthetic animations and results on open data only.

Why we like these rules

A method fitted to one match’s answers tells you about that match. A method that never saw them, and still holds up on a different league’s labels, tells you about football. The constraint that looks strictest is the one that makes the result worth trusting.

Our read

These rules reward the same things a club's data department needs: methods that generalise because they were never fitted to the fixture, and pipelines that can be audited and rerun. The system is built around them.

Sources

  1. RULES (organisers' repository)github.com
  2. FAQ (organisers' repository)github.com
  3. SUBMISSION (organisers' repository)github.com
  4. EVALUATION (organisers' repository)github.com