Enea Monzio Compagnoni

Research Intern at Google (Intrinsic) for Model Optimization.

PhD in Deep Learning @ the University of Basel.

Past: Flexion Robotics, Yahoo! Research.

[Enea Monzio Compagnoni @EneaMC]

Cool! We derived an AdamW batch-size scaling rule in Lemma 3.13 of https://t.co/zeTw8E1KtQ

B → B × k

𝗟𝗥 → 𝗟𝗥 × sqrt(𝗸)
𝗪𝗗 → 𝗪𝗗 × sqrt(𝗸)

βᵢ → 1 − sqrt(k) × (1 − βᵢ)

This was subsequently validated empirically in an ICLR 2026 paper:

https://t.co/JFC7D9LDzF

[Tatsunori Hashimoto @tatsu_hashimoto]

I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.

First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓

This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue

Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.

Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.

Which recipe gives the lowest final validation loss?

A. Keep LR and WD unchanged

B. LR ×√8

C. LR ×8

D. WD ×8

E. LR ×√8 and WD ×√8

What would you predict, and why?

[Enea Monzio Compagnoni @EneaMC]

@ExplixitAI Thanks, this paper is relevant! But where does it show that square-root beta scaling degrades under heavy-tailed noise? I'm looking for evidence of that specific claim of yours...

[Enea Monzio Compagnoni @EneaMC]

Cool! We derived an AdamW batch-size scaling rule in Lemma 3.13 of https://t.co/zeTw8E1KtQ

B → B × k

𝗟𝗥 → 𝗟𝗥 × sqrt(𝗸)
𝗪𝗗 → 𝗪𝗗 × sqrt(𝗸)

βᵢ → 1 − sqrt(k) × (1 − βᵢ)

This was subsequently validated empirically in an ICLR 2026 paper:

https://t.co/JFC7D9LDzF

[Tatsunori Hashimoto @tatsu_hashimoto]

I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.

First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓

This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue

Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.

Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.

Which recipe gives the lowest final validation loss?

A. Keep LR and WD unchanged

B. LR ×√8

C. LR ×8

D. WD ×8

E. LR ×√8 and WD ×√8

What would you predict, and why?

[Hossein Mobahi @TheGradient]

Excited to share about what I’ve been working on over the past year: quantifying the capacity of a neuron😅

This led to a mathematical framework we call HOPE, which lets us rigorously deconstruct what deep networks might have learned. Paper: https://t.co/bDn03zBkS0

[Enea Monzio Compagnoni @EneaMC]

@ExplixitAI I'm confused. I don't see either Transformers or Adam in the cited paper. Could you double-check the source and point me to the specific results supporting your claim?

[Enea Monzio Compagnoni @EneaMC]

@ExplixitAI Could be! I work under the assumption that this is not happening, essentially, so... Could be!

Do you have some paper I can look into where this is explored?

[Enea Monzio Compagnoni @EneaMC]

Btw, this was joint work with the formidable @ruuustem_10 and @orvieto_antonio, who have lately become Scaling Law Overlords:

https://t.co/srEjb5FwFS

https://t.co/7uijctaQbe

And @tianlinliu0121, whose expertise was invaluable in successfully validating our theory!

[Tatsunori Hashimoto @tatsu_hashimoto]

I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.

First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓

This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue

Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.

Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.

Which recipe gives the lowest final validation loss?

A. Keep LR and WD unchanged

B. LR ×√8

C. LR ×8

D. WD ×8

E. LR ×√8 and WD ×√8

What would you predict, and why?

[Nikita Rudin @rdn_nikita]

New model coming soon

[Rustem @ruuustem_10]

Giving a talk today at @MistralAI optimization seminar. Many thanks to @konstmish for organizing the series of talks and inviting me. I will talk about our recent works on hyperparameter scaling laws for LLMs. Looking forward to fruitful discussion with the team

[Enea Monzio Compagnoni @EneaMC]

@ruuustem_10 @MistralAI @konstmish Can you please share the slides? I am curious!

[Spencer Frei @sfrei_]

I'm hiring a Student Researcher to work on scaling laws at Google DeepMind! Project is for 19 weeks, starting winter/spring '27 (ideally: January '27), in-person in downtown SF. If you're interested, fill out this form: https://t.co/fh4Vzp0YnN

[Rustem @ruuustem_10]

Happy to share that I co-authored two papers accepted to NeurIPS: one on local SGD: https://t.co/pxAormnSxB and one on Adam-SGD gap: https://t.co/BXuedzJlNR. Thanks a lot to all collaborators!

[Flexion @FlexionAI]

We're excited to be partnering with @odysseyml to bring foundation world models into physical applications.

Here's a first look at the world action model we've been training.
Read more about our collaboration on their blog.

[Flexion @FlexionAI]

We have been named the Hottest Robotics Startup at the TOP 100 Swiss Startup Awards.

Building general-purpose intelligence that can adapt to any robot requires exceptional people working across every layer.

Congratulations to the entire Flexion team for teaching robots how to think, move and adapt.

This is just the beginning.

#GeneralRobotIntelligence #SwissStartups

[Anderson Cooper 360° @AC360]

Former Anthropic employee Jacob Coxon says the AI industry is "gambling with our lives." He tells Anderson what he finds "most scary is if AI is used to make itself more intelligent" and warns these companies are "compelled to race toward building a deadly technology."

[Nikita Rudin @rdn_nikita]

Great collaboration with @NianticSpatial

[Tony S.F. @tonysilveti]

it's nice that they are honest about just studying a quadratic model instead of what i usually see, which is people using jargon to hide that their model is overly simplified (quadratic).

[Enea Monzio Compagnoni @EneaMC]

@orvieto_antonio @tonysilveti big hug

[Flexion @FlexionAI]

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem.

Partnering with @NianticSpatial and @NVIDIARobotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user.

The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments.

#HumanoidRobots #Flexion #NianticSpatial #NVIDIA