Jonas Spinner

PhD student in high energy physics

[Jonas Spinner @jonas_spinner]

Excited to share the Lorentz-equivariant Geometric Algebra Transformer (L-GATr) for high-energy physics.

w/ @victorbreso @pimdehaan @jessethaler Tilman Plehn and @johannbrehmer
https://t.co/CeTdN2sNka

[Javier Mariño Villadamigo @javivilladamigo]

Can transformers learn the universal pattern of jet radiation and extrapolate beyond training data?

Preprint
'Extrapolating Jet Radiation with Autoregressive Transformers' in
https://t.co/vHYTLeHpZ1

w @jonas_spinner, Ayo Ore, François Charton, Anja Butter and Tilman Plehn
1/7

[Jonas Spinner @jonas_spinner]

Thanks to the L-GATr team @victorbreso @pimdehaan Tilman Plehn Huilin Qu @jessethaler @johannbrehmer

Looking forward to exciting discussions at NeurIPS!

7/7

[Jonas Spinner @jonas_spinner]

For the first time, we have trained a Lorentz-equivariant architecture on a real-world tagging dataset (JetClass = 100M events). We find the hierarchy GNN < transformer < Lorentz-equivariant transformer, showing that equivariance also matters at scale.

5/7

[Jonas Spinner @jonas_spinner]

We train continuous normalizing flows with Riemannian flow matching and several choices for the vector field architecture, and compare them with our autoregressive density estimator 'JetGPT'. CNFs turn out to be more data-efficient, and turning them equivariant also helps.

6/7

[Jonas Spinner @jonas_spinner]

We implement the L-GATr attention as a multiplicative list of signs for the queries in the inner product, and then use off-the-shelf attention kernels. With this trick, L-GATr scales to many tokens like standard transformers.

4/7

[Jonas Spinner @jonas_spinner]

Thrilled to announce that L-GATr is going to NeurIPS 2024! Plus, there is a new preprint with extended experiments and a more detailed explanation.

Code: https://t.co/FwPJ28J0DJ
Physics paper: https://t.co/SnFa0lFJZe
CS paper: https://t.co/CeTdN2sNka

1/7

[Jonas Spinner @jonas_spinner]

To build L-GATr, we replace each transformer module with a version that processes geometric algebra objects in a Lorentz-equivariant way. Plus, there is a new operation in geometric algebra that allows for an extra layer, the geometric product.

3/7

[Jonas Spinner @jonas_spinner]

The Lorentz-Equivariant Geometric Algebra Transformer (L-GATr) uses spacetime geometric algebra to process particles at the LHC in a Lorentz-equivariant way. We process them using a transformer architecture, combining the benefits of Lorentz and permutation equivariance.

2/7

[Jonas Spinner @jonas_spinner]

Interested in using L-GATr (Lorentz-equivariance + geometric algebra representations + transformer) for your own high-energy physics application? Check out the L-GATr codebase at https://t.co/FwPJ28J0DJ

[Jonas Spinner @jonas_spinner]

Excited to share the Lorentz-equivariant Geometric Algebra Transformer (L-GATr) for high-energy physics.

w/ @victorbreso @pimdehaan @jessethaler Tilman Plehn and @johannbrehmer
https://t.co/CeTdN2sNka

[Jonas Spinner @jonas_spinner]

L-GATr is as good as or better than SOTA on regression, classification, and generation tasks from particle physics. And there are many more problems in particle physics that L-GATr could help with.

[Jonas Spinner @jonas_spinner]

Excited to share the Lorentz-equivariant Geometric Algebra Transformer (L-GATr) for high-energy physics.

w/ @victorbreso @pimdehaan @jessethaler Tilman Plehn and @johannbrehmer
https://t.co/CeTdN2sNka

[Jonas Spinner @jonas_spinner]

We build the (to the best of our knowledge) first Lorentz-equivariant generative network. It uses Riemannian Flow Matching (https://t.co/BH2ozNPGW2) to hard-code phase space boundaries into the choice of trajectories.

[Ricky T. Q. Chen @RickyTQChen]

Excited to share our new work on Riemannian Flow Matching.

Unlike diffusion-based approaches, it’s
- completely simulation-free on simple manifolds,
- trivially applies to higher dimensions,
- tractably generalizes to general geometries!

https://t.co/sO2nSQrTjZ

w/ @lipmanya