Mansheej Paul

@periodiclabs

[Mansheej Paul @mansiege]

Check out our new work: Critique-out-Loud (CLoud) reward models where we improve reward models by having them generate a critique for a response before scoring it. Results and details in thread from @ZackAnkner.

[Zack Ankner @ZackAnkner]

Excited to announce our new work: Critique-out-Loud (CLoud) reward models. CLoud reward models first produce a chain of thought critique of the input before predicting a scalar reward, allowing reward models to reason explicitly instead of implicitly!

https://t.co/CnYEDM36no

[Prithviraj (Raj) Ammanabrolu @rajammanabrolu]

100 citations for a paper that taught us many lessons esp about branding, timing, peer review, and pushing the frontier! I'm rather proud of this one. Congrats to @ZackAnkner and @mansiege!
https://t.co/GO0phpdWlM

[Mansheej Paul @mansiege]

The next frontier of AI is where it meets the physical world, generates new hypotheses, and learns from experiments. Excited to join an incredible team in accelerating science and pushing this frontier.

[Liam Fedus @LiamFedus]

Today, @ekindogus and I are excited to introduce @periodiclabs.

Our goal is to create an AI scientist.

Science works by conjecturing how the world might be, running experiments, and learning from the results.

Intelligence is necessary, but not sufficient. New knowledge is created when ideas are found to be consistent with reality. And so, at Periodic, we are building AI scientists and the autonomous laboratories for them to operate.

Until now, scientific AI advances have come from models trained on the internet. But despite its vastness — it’s still finite (estimates are ~10T text tokens where one English word may be 1-2 tokens). And in recent years the best frontier AI models have fully exhausted it.

Researchers seek better use of this data, but as any scientist knows: though re-reading a textbook may give new insights, they eventually need to try their idea to see if it holds.

Autonomous labs are central to our strategy. They provide huge amounts of high-quality data (each experiment can produce GBs of data!) that exists nowhere else. They generate valuable negative results which are seldom published. But most importantly, they give our AI scientists the tools to act.
We’re starting in the physical sciences.

Technological progress is limited by our ability to design the physical world.

We’re starting here because experiments have high signal-to-noise and are (relatively) fast, physical simulations effectively model many systems, but more broadly, physics is a verifiable environment. AI has progressed fastest in domains with data and verifiable results - for example, in math and code. Here, nature is the RL environment.

One of our goals is to discover superconductors that work at higher temperatures than today's materials. Significant advances could help us create next-generation transportation and build power grids with minimal losses. But this is just one example — if we can automate materials design, we have the potential to accelerate Moore’s Law, space travel, and nuclear fusion.

We’re also working to deploy our solutions with industry. As an example, we're helping a semiconductor manufacturer that is facing issues with heat dissipation on their chips. We’re training custom agents for their engineers and researchers to make sense of their experimental data in order to iterate faster.

Our founding team co-created ChatGPT, DeepMind’s GNoME, OpenAI’s Operator (now Agent), the neural attention mechanism, MatterGen; have scaled autonomous physics labs; and have contributed to some of the most important materials discoveries of the last decade. We’ve come together to scale up and reimagine how science is done.

We’re fortunate to be backed by investors who share our vision, including @a16z who led our $300M round, as well as @Felicis, DST Global, NVentures (NVIDIA’s venture capital arm), @Accel and individuals including @JeffBezos , @eladgil , @ericschmidt, and @JeffDean. Their support will help us grow our team, scale our labs, and develop the first generation of AI scientists.

[Mansheej Paul @mansiege]

@typedfemale Well... only if we happen upon space cocaine in the sand...

[Cody Blakeney @code_star]

It was kinda a movie

[Mansheej Paul @mansiege]

Imagine if memory pointers had twitter. They’d be like “@malloc is this true?”

[Epimenid @truthful_cretan]

Imagine if Linux kernel interfaces had twitter. They’d be like “/proc is this true?”

[Misha Laskin @MishaLaskin]

Engineers spend 70% of their time understanding code, not writing it.

That’s why we built Asimov at @reflection_ai.

The best-in-class code research agent, built for teams and organizations.

[Mansheej Paul @mansiege]

Imagine if threads had twitter. They’d be like “@lock can I do?”

[Mansheej Paul @mansiege]

Imagine if boats had twitter. They’d be like “@dock is this true?”

[Mansheej Paul @mansiege]

Imagine if boats had twitter. They’d be like “@dock is this true?”

[Cody Blakeney @code_star]

Imagine if soup had twitter. They'd all be like "@stock is this true?"

[Davis Blalock @davisblalock]

Deep learning training is a mathematical dumpster fire.

But it turns out that if you *fix* the math, everything kinda just works…fp8 training, hyperparameter transfer, training stability, and more. [1/n]

[Mansheej Paul @mansiege]

@code_star Tbh, sounds like a great guy!

[Mansheej Paul @mansiege]

@nsaphra You should check out Victor Pelevin. Omon Ra is great!

[Mansheej Paul @mansiege]

@nsaphra This is such an awesome book. Everything by these authors honestly.

[Mansheej Paul @mansiege]

@code_star https://t.co/YMBWsOwaqe

[Mansheej Paul @mansiege]

@rajammanabrolu I thought grpo is agi now?

[Dan Biderman @dan_biderman]

How can we use small LLMs to shift more AI workloads onto our laptops and phones?

In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@openai, @together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost while maintaining 97.9% of the accuracy.

See Gru and the Minions in action below, 🔉on please (h/t @cartesia)!

[Core Francisco Park @corefpark]

💥New Paper!
Algorithmic Phases of In-Context Learning:

We show that transformers learn a superposition of different algorithmic solutions depending on the data diversity, training time and context length!

1/n

[Zack Ankner @ZackAnkner]

Critique out loud reward models made it into the Kimi k1.5 technical report! Super cool to see someone scale it up to 800k inputs and to see how much better reward modeling it led to!

[Cody Blakeney @code_star]

If you want to read more about the curriculum training used in OLMo 2 checkout our (@mansiege @_BrettLarsen Sean Owen) paper!

Congrats on the release to everyone at AI2! (but especially @soldni and @kylelostat <3 data )

https://t.co/e0V5B4TxTS

[Zack Ankner @ZackAnkner]

Agreed ;)

But in all seriousness, its cool to see everyone converging on reward models that perform explicit reasoning by critiquing out loud. Super excited to see how people build on top of these works.

[Mansheej Paul @mansiege]

Code and models for our latest work Critique-out-Loud (CLoud) Reward models is now released! Check out our paper (https://t.co/SQOQYGe27y) for more details on using reward models to reason before predicting a reward score.

[Zack Ankner @ZackAnkner]

Code and models for Critique-out-Loud (CLoud) reward models are finally public! The repo comes with a gradio demo you can run, so hopefully people can mess around with the models 😃

Code: https://t.co/oJJgC5M67f