Shamane Siri | Pluralis

Tinkering Transformers | Coding by Day, Hallucinating by Night

[Shamane Siri | Pluralis @GShamane]

Agentic RL environments are becoming critical. We integrated OpenReward (https://t.co/PGK3Ilhj1C) into Alibaba’s ROLE (https://t.co/3SkXlH5hRZ).

Details: https://t.co/BWR9lZ0dcJ

[Thalaiyasingam Ajanthan @tha_ajanthan]

On the back of our ICML workshop on Protocol Learning, our official workshop proposal has been accepted at NeurIPS, titled "Open, Collaborative, and Decentralized Training of Foundation Models". There is an exceptional set of speakers, and the call for papers is coming soon!
In collaboration with @benjamintherien, @Ana_koloskova, @ebelilov, @niclane, #kaja, #Aakanksha, @Pluralis

[Shamane Siri | Pluralis @GShamane]

What we actually need: dense signals with built-in exploration. Enough bits per step to learn, enough exploration to reach past the base, without erasing what's already there.

Really bullish on on-policy distillation when you have a powerful model, and of course model merging.

[Shamane Siri | Pluralis @GShamane]

RL alone isn't enough for domain adaptation and continual learning.

Time to rethink domain adaptation + continual learning for 100B+ models.

[Shamane Siri | Pluralis @GShamane]

RL still has a major part: trained on outcomes it generalizes and regularizes, where SFT tends to memorize and drift. But it's sparse and mostly sharpens what the base already knows.

[Shamane Siri | Pluralis @GShamane]

Self Supervised Learning !

[Thalaiyasingam Ajanthan @tha_ajanthan]

Thoroughly enjoyed the talk by @konstmish at our workshop today. @Pluralis
Happening now, schedule here: https://t.co/770T7THbCR

[Shamane Siri | Pluralis @GShamane]

Very interesting to see this development.

Sensor Data --> is the bridge where agents can connect with the physical layer.

[Google Research @GoogleResearch]

Introducing SensorFM, a large-scale Sensor Foundation Model that learns from 1 trillion-minutes of unlabeled wearable data drawn from five million consented participants.

SensorFM learns a single, reusable representation of sensed human physiology that transfers across cardiovascular, metabolic, sleep, and mental health, as well as lifestyle and demographic factors.

More →https://t.co/lbi1DG0zAW

[Shamane Siri | Pluralis @GShamane]

But one important thing, there a rotation happening , but for a given window it can act as pretty solid single direction .

[Shamane Siri | Pluralis @GShamane]

Similar line of paper, but claims RLVR raw weights move in a curve , but it only has a rank 1 dominant direction and it moves pretty linearly.

https://t.co/NHAVkqwXYP

[Shamane Siri | Pluralis @GShamane]

Been reading a lovely paper, "Linear Dynamics in the RLVR Training of LLMs" (Wang et al.): during RL, model weights move almost in a straight line over training.

We got curious how this looks on a weaker, different base model. A short exploration. ��

[Shamane Siri | Pluralis @GShamane]

Module wise.

[Shamane Siri | Pluralis @GShamane]

@michael_jupp4 we did try second-order curvature fits, both fixed and adaptively refit. They did worse, not better, and when free to choose, the fit just zeroed the curvature term.

[Shamane Siri | Pluralis @GShamane]

Been reading a lovely paper, "Linear Dynamics in the RLVR Training of LLMs" (Wang et al.): during RL, model weights move almost in a straight line over training.

We got curious how this looks on a weaker, different base model. A short exploration. 🧵

[Shamane Siri | Pluralis @GShamane]

Takeaway: the linear regime is real, and its strength tracks base-model capability vs task.

Modern pipelines distil + SFT first, then RL, which keeps the model capable and its RL weights linear. Exactly where these extrapolation / compression ideas should pay off. Summary 👇

[Shamane Siri | Pluralis @GShamane]

Nicely, this matches the paper's own intuition: they anticipate piecewise linearity under distribution shift, and expect their lazy-training view to hold while the model stays near its starting region.

A weaker model on harder math just sits closer to that edge.