04Solutions family

Vision and calibration

Detectors, trackers and calibrators that rebuild players, ball and camera from video.

Where it starts: boxes on the players, and a much smaller one on the ball.
  • 10entries
  • 3calibrators
  • 5licence families in play

One picture: three stages, one error budget

Boxes in the image become tracks over time, and tracks become positions on the pitch. Each stage hands its mistakes to the next.

Video becomes tracking in three moves. Detect finds players and ball in each frame. Track links detections over time and tries to keep identities. Calibrate estimates the camera, so that image positions become pitch positions. Every entry in this family does one of the three, or all of them at once.

Seeing is needed when there is no tracking feed, or when the feed has holes that video could fill.

How to choose inside it

Entry Labels What it needs in view Compute
Roboflow soccer detectors Pretrained Players and a visible ball GPU, every frame
Tracking by detection Pedestrian weights Detections from a football detector Light on top of a detector
SportsLabKit Set by the detector A full-pitch view suits it best Set by the detector
SAM 2 None, but a prompt per object The prompted object GPU, per object
CoTracker None, but query points The queried points GPU
SoccerNet re-identification Football identity labels Player crops with enough pixels GPU
PnLCalib Pretrained Pitch lines and keypoints GPU, per frame
TVCalib Pretrained segmenter Pitch markings Iterative, per frame
KpSFR Pretrained Pitch in view GPU
Game State Reconstruction baseline Pretrained baseline All of the above Heavy GPU

Choose by camera first. A fixed tactical camera wants motion-only tracking and one calibration for the match. A broadcast camera wants camera-motion compensation, calibration on every frame and the shot handling of the events and replay family. The calibrators are graded on the SoccerNet Camera Calibration benchmark.

A fixed tactical camera holds the whole pitch: one calibration, no cuts, and a very small ball. A broadcast camera pans across part of it and needs a new calibration on every frame.

What this family cannot do

  • It cannot make the ball easy. A football is a few fast, blurred pixels, and it is lost most often when possession is contested.
  • It cannot tell team-mates apart reliably. Identical kits defeat appearance, and identity is what player-level possession needs.
  • It cannot give height from one camera. A ball in the air projects to the wrong place on the ground.
  • It cannot ship everywhere. MIT, Apache-2.0, GPL, AGPL and CC BY-NC all appear in this family.

Where our decoder stands

Downstream of this family, and independent of it. The decoder consumes a tracking feed and automatic tags and asks nothing of the video. It was built for missing and noisy evidence, which is the regime video-derived tracking produces. The output of this family is the kind of input it is designed to take.

In this family10

  • Three YOLOv8 checkpoints for players, ball and pitch keypoints, wired into a runnable pipeline with team clustering and a radar view.

    • Video
    • Detection
    • Open weights
    • Mixed licences
  • The full chain from broadcast video to a minimap: detect, track, identify team, role and shirt number, and place every athlete on the pitch.

    • Video
    • Full pipeline
    • Open weights
    • GPL-3.0
  • Three ways to link per-frame detections into tracks. They differ in what they trust: boxes alone, boxes plus camera motion, or boxes plus appearance.

    • Video
    • Multi-object tracking
    • Open weights
    • MIT / GPL-3.0
  • A toolkit that turns sports video into tracking tables: pluggable detectors, motion and appearance models, and conversion to pitch coordinates.

    • Video
    • Tracking toolkit
    • Full-pitch view
    • GPL-3.0
  • The benchmark and baseline for recognising the same player across camera views, in a sport where every team-mate wears the same shirt.

    • Video
    • Identity
    • Benchmark
    • MIT
  • Promptable video segmentation. Indicate an object once and get its mask in every frame, with a memory that survives occlusion.

    • Video
    • Segmentation
    • Open weights
    • Apache-2.0
  • Tracks any set of points through a video jointly, so that points help locate one another through occlusion.

    • Video
    • Point tracking
    • Open weights
    • CC BY-NC
  • Camera calibration from pitch keypoints and lines together, refined by non-linear optimisation. The method to start from when turning a broadcast frame into pitch coordinates.

    • Video
    • Calibration
    • Open weights
    • GPL code
  • Calibration as direct optimisation: segment the pitch markings, then fit the camera whose projection of the pitch model lands on them.

    • Video
    • Calibration
    • Open weights
    • MIT
  • Keypoint-based sports field registration: predict a grid of pitch keypoints, then fit the homography. The established baseline from the World Cup benchmark.

    • Video
    • Calibration
    • Open weights
    • MIT