04Classifiers and decoders

Linear-chain CRF (pytorch-crf)

Learned transition scores between adjacent labels, trained jointly with the network beneath and decoded exactly with Viterbi.

  • Sequence
  • Structured output
  • Needs labels
  • MIT
Emissions score each frame, learned transitions score each step, and the best path is decoded exactly.
Modality
Per-frame class scores from any network
Output
One coherent label path
Evidence
Lafferty et al. (2001); maintained PyTorch layer
Code and weights
pytorch-crf layer; transitions are learned per task
Licence
MIT

A conditional random field scores a whole label sequence, not each label alone. In the linear-chain form the score is a sum of two parts. Emissions say how well each frame’s features fit its label. Transitions say how compatible each label is with the next. The probability of a sequence is its score normalised over all possible sequences, which dynamic programming computes exactly.

pytorch-crf implements this as a layer. Put it on top of any network that outputs per-frame scores. Training maximises the likelihood of the true sequence, so gradients flow into both the transition matrix and the network. Inference is Viterbi.

The difference from a hidden Markov model is direction. An HMM models how states generate observations. A CRF models labels given observations, so emissions can use overlapping, arbitrary features with no independence assumption.

Where it wins

  • Transitions are learned from data, not set by hand.
  • Joint training. The network learns emissions that suit the decoder it will be decoded by.
  • Exact inference in both training and decoding.
  • Hard constraints are trivial: forbid a transition by fixing its score at minus infinity.

What it costs

  • Learning transitions requires labelled sequences.
  • First-order memory. Leaving a state costs the same after two frames or two hundred.
  • One transition matrix for all time. Context-dependent switching, cheap at a touch and expensive mid-dribble, needs an extension.
  • Three labels give nine transition scores. With so few parameters, learning them buys little over setting them well.

How it plugs into possession

It is the supervised sibling of our decoder: the same chain and the same Viterbi inference. The only difference is where the numbers come from. With possession labels, a CRF layer on a temporal convolution or boosted-tree emission model is the textbook design, and PathCRF is a far richer version of the idea.

Without labels, the structure survives and the learning goes. Fix the transition costs, compute emissions from football evidence, and decode. That is the system we fielded, described under Viterbi decoding.

Our read

The textbook structured output layer. With three states its real gift is the structure, and the structure is free.

Sources

  1. pytorch-crf: linear-chain conditional random field in PyTorch (repository)github.com
  2. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data (Lafferty, McCallum, Pereira, 2001)cs.columbia.edu