Marco Berlot

Señor Engineer at @GoogleDeepMind

Solving intelligence, one layer at a time.

[Marco Berlot @MarcoBerlot]

👋 Hello! I've, been working on something quite exciting over the past year at #GoogleDeepMind and I can now share part of it with all of you.

"#Orbax: Distributed Checkpointing with #JAX" describes how at Google we have designed a standardized checkpointing solution for high-performance distributed ML systems.

How do you enable fast recovery after a preemption? Especially for massive models? And how about resharding? We often think about how to optimise the transferring of data from host to accelerators. But how about storage?

Highly recommended read. And the best thing? The library is open source 🎉

https://t.co/NBJ1UbyVe4

[Marco Berlot @MarcoBerlot]

@dhh Your whole thesisis is built on a comparison between an ethnic minority to a group of animals.
Do I need to remind you why this is bad, and how badly it went last time we did this in Europe

[Marco Berlot @MarcoBerlot]

We're going big this time

[Logan Kilpatrick @OfficialLoganK]

We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress : )

[Marco Berlot @MarcoBerlot]

So proud for choosing to never learn regex

[Marco Berlot @MarcoBerlot]

@dhh How audacious to talk about remigration from an immigrant living in a country that greatly benefitted from immigration.

[Marco Berlot @MarcoBerlot]

@dhh "Leading by example" only works if the example actually works. No developing nation is going to sacrifice its GDP and comfort just to get a pat on the back from Brussels.

[Marco Berlot @MarcoBerlot]

@MrBeast 6,500,000

[Marco Berlot @MarcoBerlot]

@evan_thayer those portions are tiny

[Marco Berlot @MarcoBerlot]

@antirez What do you mean by "culture"? And why does it matter

[Marco Berlot @MarcoBerlot]

@connorpaton Always on the same side of people going in the opposite direction

[Marco Berlot @MarcoBerlot]

@gavinrbrown1 A local maximum hill

[Marco Berlot @MarcoBerlot]

@Pirat_Nation can you elaborate on how this contributes to thousands of tonnes of carbon emissions globally? How's this different than downloading any other thing as part of a software you are using

[Marco Berlot @MarcoBerlot]

@haileyhmt What are you working on?

[Marco Berlot @MarcoBerlot]

@ivanburazin GChat is the dive bar of corporate comms: no frills, slightly depressing lighting, and you’re only there because you’re already in the building. Slack is a $20 cocktail in a glass shaped like a bird; it’s fun until you realize you’re paying for the decor with your focus

[Marco Berlot @MarcoBerlot]

@tszzl I once saw a man stare at a sunset and try to double-tap the horizon to save it

[Marco Berlot @MarcoBerlot]

@rakyll if you’ve done it right, they’ll think you’ve done nothing at all

[Marco Berlot @MarcoBerlot]

@sama If it’s just fast and cheap, it’s a better tool for the tasks we already do. If it’s smarter, it starts doing the tasks we don't even know how to describe yet.

[Marco Berlot @MarcoBerlot]

@The_AI_Investor Gemini 2M context?
Even Anthropic uses TPUs?
Search growing 19% YoY?
Only company owning full stack?