04Classifiers and decoders
Set encoders (Deep Sets, Set Transformer)
Networks whose output cannot depend on the order of their inputs. The correct way to feed anonymous players to a model.
- Tracking
- Learned
- Order-invariant
- Needs labels
- Modality
- An unordered set of players per frame
- Output
- Frame or team embedding for a downstream model
- Evidence
- NeurIPS 2017 (Deep Sets); ICML 2019 (Set Transformer)
- Code and weights
- Authors' research code; no football weights
- Licence
- Apache-2.0 (Deep Sets), MIT (Set Transformer)
Tracking delivers a frame as a list of players, and the order of that list is an accident of the tracker. A model that reads positions as a fixed vector learns the accident. Set encoders remove it by construction.
Deep Sets shows that the simple form is enough in principle. Apply the same network to every element, combine the results with a symmetric operation such as a sum or a mean, then transform the pooled vector. Every player is encoded identically, and the result is the same whatever order they arrive in.
Set Transformer replaces independent encoding with attention. Players attend to one another before pooling, so the representation can express relations: this defender is goal-side of that attacker. Induced set attention keeps the cost manageable, and learned attention pooling replaces the plain sum.
| Deep Sets | Set Transformer | |
|---|---|---|
| Interactions | Only through pooling | Explicit, through attention |
| Cost in players | Linear | Quadratic, reduced by inducing points |
| Appetite for data | Low | Higher |
Where it wins
- Slot-order invariance is guaranteed, not learned from augmentation.
- Variable set size is natural. Ten players or eleven, the same network.
- Deep Sets is tiny and hard to overfit. Set Transformer captures pairwise structure when data allows.
- Team-only data fits exactly: two sets, home and away, no identities required.
What it costs
- Pooling loses information. A sum-pooled model struggles with “the nearest opponent to the ball”, which is a relation between elements.
- Invariance to order is not robustness to absence. A missing or duplicated player still shifts the pooled vector.
- They are encoders, not solutions. Training needs possession labels, and time has to be modelled elsewhere.
- Attention on small labelled sets overfits readily.
How it plugs into possession
As the front end of any learned possession model. Borghesi’s TCN uses the pattern, and PathCRF uses set attention for the same reason. For a feed where players carry a team and no identity, a set encoder is not an optimisation. It is the only input layer that respects the data.
The principle applies without a network too. Hand-built evidence made of minima, counts and means over a team, such as nearest player, players within a radius or mean speed, is permutation-invariant by design.
Our read
Not optional for anonymous players. Start with Deep Sets and move to attention when labels are plentiful.