Maximo Rulli

Studying how machines learn 🤖 and teaching them some things 🧠⚙️
CS @SapienzaRoma

[Maximo Rulli @MaximoRulli]

Diffusion language models develop 𝐬𝐮𝐛𝐥𝐢𝐦𝐢𝐧𝐚𝐥⏱️

Despite receiving no explicit timestep, mask-based DLMs encode denoising progress in a low-dimensional latent structure. We can steer this clock predictably and watch the model correct perturbations across layers.

(1/6)

[Maximo Rulli @MaximoRulli]

Read the full details in our paper!

Preprint: https://t.co/JHNq8C6Xmr

Big thanks to the amazing team: @tvfontanari @simone_petruzzi @FedericoA45200 @krytox205 @DonatoCrisosto1 @GiorgosNik02 @tommaso_mncttn @teelinsan @EmanueleRodola @s_scardapane @devoto_alessio

(6/6)

[Maximo Rulli @MaximoRulli]

Diffusion language models develop 𝐬𝐮𝐛𝐥𝐢𝐦𝐢𝐧𝐚𝐥⏱️

Despite receiving no explicit timestep, mask-based DLMs encode denoising progress in a low-dimensional latent structure. We can steer this clock predictably and watch the model correct perturbations across layers.

(1/6)

[Maximo Rulli @MaximoRulli]

The latent time signal is strongly low-dimensional.

Across layers, one principal component often captures >90% of the variance, while the top two recover nearly the full steering effect. In 2D/3D, the trajectory forms an ordered, manifold-like curve.

(5/6)

[Maximo Rulli @MaximoRulli]

@paradigmainc amazing

[OmnAI Lab @OmnAI_Lab]

1/ Large Language Models leak energy when they hallucinate. We built a training-free method to catch the spill and keep them *grounded*.

Our #ICLR2026 paper introduces Spilled Energy for SOTA zero-shot detection.

TLDR: Hallucinations violate the probability chain rule.

[Simone Scardapane @s_scardapane]

*Attention Sinks in Diffusion Language Models*
by @MaximoRulli @devoto_alessio @simone_petruzzi @fabreetseo

We study attention sinks in diffusion models, with many findings including "moving" sinks and a strong robustness of the underlying models.

https://t.co/217DMDDVGJ

[Maximo Rulli @MaximoRulli]

What do a 5-year-old and a Diffusion Language Model have in common 👦🤖?

Neither can keep their attention in one place!

We show that DLMs shift their attention sink tokens across denoising steps

This anchor flexibility makes the models more robust when masking sink tokens👇👇

[Alessio Devoto @devoto_alessio]

Attention sinks emerge in Diffusion LLMs too, but they move around! 👀 We find that: sinks shift positions during generation & Diffusion models are surprisingly robust to masking them! ⚡️ w/ @MaximoRulli @simone_petruzzi @s_scardapane @fabreetseo

[Alessio Devoto @devoto_alessio]

Attention sinks emerge in Diffusion LLMs too, but they move around! 👀 We find that: sinks shift positions during generation & Diffusion models are surprisingly robust to masking them! ⚡️ w/ @MaximoRulli @simone_petruzzi @s_scardapane @fabreetseo