Roman Vorushin

Research Engineer at Google DeepMind | ex-Grammarly | AI/ML since 2011

[Roman Vorushin @vorushin]

@teortaxesTex Le Chaton is superhuman smart, but sleeps the large part of the day and follows no instructions.

[Roman Vorushin @vorushin]

There is no “it’s just a noise” in frontier training.

[rohan anil @_arohan_]

Every loss spike is your model telling you something. Listen carefully

[Roman Vorushin @vorushin]

Surprise! Big Corps want to train and own their Minds

[Roman Vorushin @vorushin]

Lakes: Zurisee, Zugersee, Vierwaldstättersee

[Roman Vorushin @vorushin]

@giffmana Zugersee?

[Roman Vorushin @vorushin]

@giffmana @sshkhr16 Had no doubts about the author of big_vision :)

I heard the same from people that went GMD -> A\. Were missing tools (XManager FTW), but quickly built better versions.

[Roman Vorushin @vorushin]

@giffmana @sshkhr16 It’s the best time to rewrite useful tech one had at Google.

System design interviews should be all about it.

[Roman Vorushin @vorushin]

@giffmana @Dorialexander I would love to see a map of GPUs

[Roman Vorushin @vorushin]

@giffmana Sihlsee and Einsideln in one picture with Zurisee.

We just need more GPUs/TPUs!

[Roman Vorushin @vorushin]

"including foreign national Anthropic employees"

[Roman Vorushin @vorushin]

- Hey, agentic AI, how not to get dumber while using you?
- Easy, just interview a lot!

[Roman Vorushin @vorushin]

@DimitrisPapail AI that is beneficial for everyone

[Roman Vorushin @vorushin]

@DimitrisPapail user: "you did great job. next step - 25kB target. go ahead, you have 10 hours till I wake up"

[Roman Vorushin @vorushin]

@suchenzang No LLMs can drink Flat White as I can!

[Roman Vorushin @vorushin]

At one point, Malachowsky and Priem wanted to call the company NVision, but that name was already taken by a manufacturer of toilet paper. Both Priem and Huang have taken credit for coming up with the name Nvidia, from "invidia", the Latin word for "envy".

[Roman Vorushin @vorushin]

@giffmana Did you see project Marin? https://t.co/N07g0ORCob

[Larry Dial @classiclarryd]

Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions. https://t.co/5dPB9uBiSp

[Roman Vorushin @vorushin]

After reading "The Beginning of Infinity" I maintain a list of explanations of how LLMs work. Put it online now: https://t.co/GIdgK6B7zi

[Roman Vorushin @vorushin]

"If a systems guy could dream a bit, they’d want AI architectures to directionally look like a matmul loop."

[Mark Saroufim @marksaroufim]

My MLSys keynote on AI writing systems code got more interest than I expected. The recording will take a while, so in the finest tradition of AI labs sharing blog posts, we’re starting the Core Automation Blog with this one https://t.co/h4uSOyrglf

[Roman Vorushin @vorushin]

@marksaroufim "I was told that the recording wouldn’t be available for another few months" great potential for using AI for that :)

Thanks for the great article!

[Roman Vorushin @vorushin]

@DimitrisPapail A grid. A digital frontier. I tried to picture clusters of information how they move through the computer.