Naypho

[Bartosz Sobkowicz @gersonist1096]

[9/9] Sure, it's still a simplification - it doesn't account for the positioning of the other players on the pitch, for one. But it's undeniably a better way to model continuous positions than a single distribution. Question that remains: how many components is optimal?

[Bartosz Sobkowicz @gersonist1096]

[8/9] For a simple clearance from the 6-yard box, a single Gaussian says: "Somewhere around the first-second third boundary, with a spread of a third of the pitch." Not very informative. MDN instead gives several options, ranking short/near-side passes above the rest.

[Bartosz Sobkowicz @gersonist1096]

[7/9] The results are striking. In the charts below, the star marks the player's position, and the crosses mark the centers of each component - showing where the pass might go. The redder the color, the higher the probability of a pass landing there.

[Bartosz Sobkowicz @gersonist1096]

[6/9] That's where MDN (Mixture Density Network) came in. It needs just one arbitrary input - how many components you want to model. The model then learns where the ball is likely to go next, returning both the probability and the spread (std dev) of each component.

[Bartosz Sobkowicz @gersonist1096]

[5/9] It would be great to model each scenario somewhat independently, without rigidly defining it upfront. Let the model itself learn which passes are more or less likely in a given situation.

[Bartosz Sobkowicz @gersonist1096]

[4/9] The key word for me turned out to be "scenarios." In any situation, a player has several viable scenarios: a short ball back, a pass to the nearer flank, a long vertical ball through the middle, or a switch to the other side (and more, depending on position).

[Bartosz Sobkowicz @gersonist1096]

[3/9] Imagine a player around the halfway line with two options - play it back to a defender, or forward to a striker. A single Gaussian averages both and lands on a short, safe pass with barely any movement. That's not the real dilemma a player faces.

[Bartosz Sobkowicz @gersonist1096]

[2/9] Early models based on a single Gaussian had one core issue: they averaged the situation. Which is exactly what they're supposed to do. But in football, that's not good enough - possible pass directions and destinations aren't captured by a single Gaussian.

[Bartosz Sobkowicz @gersonist1096]

[1/9] Turns out simulating where a pass from a given position might end up is not as trivial as it sounds. When I first tackled this, I thought: "Okay, we'll just fit a Gaussian on historical passes and call it a day." Not a bad idea, just not enough.

[Bartosz Sobkowicz @gersonist1096]

[8/8] For those curious, a few more Fourier vs. raw coordinate comparisons - same pattern, different starting points on the pitch.

[Bartosz Sobkowicz @gersonist1096]

[7/8] Skeptical: "Fourier looks nicer" ≠ "Fourier is better"? Fair.
I trained two identical models differing only in coordinate encoding - same architecture, same data.
Fourier: 0.76 ROC-AUC
Raw: 0.71 ROC-AUC
The encoding alone made the difference.

[Bartosz Sobkowicz @gersonist1096]

[6/8] The difference shows up directly in the data. A pass from deep, near the byline on the right — comparing xT for every possible end location in the opponent's half:
Raw coordinates favor a deep pass into the box. Fourier instead favors a cutback to the edge of the box.

[Bartosz Sobkowicz @gersonist1096]

[5/8] Fourier encoding fixes this: each coordinate is expanded into a series of frequencies (sin/cos).
Low frequencies encode the general pitch zone (half, third). High frequencies capture sub-meter precision. One coordinate, multiple resolutions at once.

[Bartosz Sobkowicz @gersonist1096]

[4/8] This is a classic "spectral bias": a network with a simple, linear projection of coordinates struggles to capture sharp, precise boundaries in space (like the penalty box line), even if it technically has full positional information.

[Bartosz Sobkowicz @gersonist1096]

[3/8] The problem: raw x/y coordinates are poorly represented by a network.
Take (110, 61) and (110, 63) - right next to each other, but one is inside the box, the other just outside. To a neural net, that's almost the same point.

[Bartosz Sobkowicz @gersonist1096]

[2/8] Fourier analysis breaks down a complex signal into a sum of simple sine waves at different frequencies - mostly known from audio and signal processing.
The same idea works surprisingly well for pitch coordinates.

[Bartosz Sobkowicz @gersonist1096]

[1/8] What do football and the Fourier transform have in common? My answer: positional encoding.
Here's how a technique from signal processing changed how my xT model encodes pitch location

[Bartosz Sobkowicz @gersonist1096]

[10/10] In upcoming posts I'll share some interesting details that came out of this project, because it was more fun than anything else I've worked on. I see plenty of limitations, but it could be a good starting point for other ideas.

[Bartosz Sobkowicz @gersonist1096]

[9/10] What's the biggest advantage of this approach? Full sequence memory - the model knows every prior event in the action. Biggest downside? This version is extremely inefficient.

[Bartosz Sobkowicz @gersonist1096]

[8/10] That contribution is how much that specific event increased (or decreased) the probability that the action ends in a goal - and that's how we can evaluate every single event in a match.