05Family 05 of 05

Proof

A design is a hypothesis until it is measured on matches it has never seen, by a rule written before the numbers came in.

Your question The study that answers it Is it actually better? Two metrics, one verdict How sure, and of what? Confidence and provenance What is still unsolved? Open frontiers
The decision map of this family. Each question leads to the study that answers it.
  • 3studies in this family
  • 3stages of selection
  • 9questions the field has not settled

How belief accumulates

No single test makes a design credible. Belief is built in gates, and each gate proves one thing and nothing more.

Four gates, each a step up: the logic is right, the dials are sane, it works on real football, and someone else gets the same number. The fixture's public window sits off the ladder: it checks the file and adds no belief about quality.

Every claim becomes a test

The other four families make claims. This one turns each claim into the test that could refute it, on the same bench.

Five claims from the other families, one bench: whole matches the design has never seen. Each claim comes back as its test.

Three kinds of proof

Question What settles it Study
Is the design better? Both official metrics, on whole held-out matches, selected by the published rule Two metrics, one verdict
Does it know when it is unsure? Three confidences kept apart, and a record of where every value came from Confidence and provenance
What has nobody proved yet? Nine open questions, each with the experiment that would close it Open frontiers

How an architect chooses here

  • The unit is the match. Neighbouring frames are near-copies. Splitting a match by frames leaks the answer.
  • Write the rule first. Select by the published ranking rule, never by a blend invented after the scores.
  • Degrade the labelled data to the target. A design validated with a ball that is always tracked has not met the problem.
  • Publish what reproduces. If someone else cannot regenerate the number from public data, it is a claim.

What this family cannot do

Proof cannot rescue a design. It can only tell the truth about one. The designs themselves start on the map.

In this family3

  • How a possession design earns belief when its two official metrics disagree and its fixture cannot be tuned on. The lenses to report, the stages of selection, and the rule that ...

    • Evaluation design
    • Pareto
    • Held-out matches
  • Three different uncertainties hide behind the word confidence: was it seen, is it the same instant, and which state explains it. Provenance keeps them apart and keeps the answer...

    • Uncertainty
    • Provenance
    • Leakage
  • Nine questions possession inference has not settled, each with the experiment that would settle it. This is where the next gains are.

    • Open questions
    • Experiments
    • Field