04Classifiers and decoders
Linear-chain CRF (pytorch-crf)
Learned transition scores between adjacent labels, trained jointly with the network beneath and decoded exactly with Viterbi.
- Sequence
- Structured output
- Needs labels
- MIT
- Modality
- Per-frame class scores from any network
- Output
- One coherent label path
- Evidence
- Lafferty et al. (2001); maintained PyTorch layer
- Code and weights
- pytorch-crf layer; transitions are learned per task
- Licence
- MIT
A conditional random field scores a whole label sequence, not each label alone. In the linear-chain form the score is a sum of two parts. Emissions say how well each frame’s features fit its label. Transitions say how compatible each label is with the next. The probability of a sequence is its score normalised over all possible sequences, which dynamic programming computes exactly.
pytorch-crf implements this as a layer. Put it on top of any network that outputs per-frame scores. Training maximises the likelihood of the true sequence, so gradients flow into both the transition matrix and the network. Inference is Viterbi.
The difference from a hidden Markov model is direction. An HMM models how states generate observations. A CRF models labels given observations, so emissions can use overlapping, arbitrary features with no independence assumption.
Where it wins
- Transitions are learned from data, not set by hand.
- Joint training. The network learns emissions that suit the decoder it will be decoded by.
- Exact inference in both training and decoding.
- Hard constraints are trivial: forbid a transition by fixing its score at minus infinity.
What it costs
- Learning transitions requires labelled sequences.
- First-order memory. Leaving a state costs the same after two frames or two hundred.
- One transition matrix for all time. Context-dependent switching, cheap at a touch and expensive mid-dribble, needs an extension.
- Three labels give nine transition scores. With so few parameters, learning them buys little over setting them well.
How it plugs into possession
It is the supervised sibling of our decoder: the same chain and the same Viterbi inference. The only difference is where the numbers come from. With possession labels, a CRF layer on a temporal convolution or boosted-tree emission model is the textbook design, and PathCRF is a far richer version of the idea.
Without labels, the structure survives and the learning goes. Fix the transition costs, compute emissions from football evidence, and decode. That is the system we fielded, described under Viterbi decoding.
Our read
The textbook structured output layer. With three states its real gift is the structure, and the structure is free.