Sergio Charles

co-founder @thesis_labs (yc f25) | automating ai research | prev. @google, @nvidia, @stanford

[Sergio Charles @eigentopology]

@thesis_labs gives you an AI researcher that runs experiments for you: train models, track runs, and iterate on results autonomously at scale.

Get $100 in free GPU credits by signing up at https://t.co/sC0x5KaskV and commenting how you'd use Thesis.

We're onboarding 20 research teams this month. Reply if you want in.

#AIResearch #MachineLearning #AI #AutoResearch

[Sergio Charles @eigentopology]

Human intelligence emerged from scarcity, not abundance. ASI will follow the same path.

[Sergio Charles @eigentopology]

@AdaFang_ This is great! A delegation of auto scientists seems like the way to invent 'novel' ideas.

[Sergio Charles @eigentopology]

Just the start

[Alex Rives @alexrives]

Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.

The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.

We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.

We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.

ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.

A world model of protein biology emerges through language modeling.

We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.

The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.

This understanding emerges without prior knowledge, just from language modeling of protein sequences.

Language models are becoming a powerful substrate to understand and program biology.

The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.

I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.

[Tiberiu Mușat @Tiberiu_Musat_]

Why does deep learning generalize? What does weight decay really do? Can algorithmic information theory address these questions?

In my latest preprint, I give a proof that the minimum neural weight norm matches the minimum program length (aka Kolmogorov Complexity), up to a logarithmic factor. In other words, the neural network with the smallest possible weight norm (that fits the data) must encode the shortest program (that fits the data).

The result only holds for fixed-precision neural nets: infinite precision nets can store infinite information with finite (small) weights.

https://t.co/eMZIGQDf2f

[Sergio Charles @eigentopology]

@wenhaocha1 @karpathy Amazing!

[Sergio Charles @eigentopology]

@Andy_ShuoYang @CShorten30 Really cool! What was your process for speeding up the operators?

[SpaceX @SpaceX]

Liftoff of Starship!

[Hongjian Zhou @itsEmZee_]

If this is not obvious enough...

> Mar 25, Sakana’s AI Scientist makes Nature
> May 13, RSI raises $650M to build self-improving AI
> May 13, Adaption ships AutoScientist for model training
> May 19, Karpathy joins Anthropic pretraining
> May 19, Nature drops 3 AI-scientist papers in one day

The next frontier is not bigger models.

It’s AI researchers building AI researchers.

[Sergio Charles @eigentopology]

@itsEmZee_ Absolutely! This is what we’re building towards at @thesis_labs

[Sergio Charles @eigentopology]

@jparkjmc @sama +1 @sama we need tokens to automate ai research

[Sergio Charles @eigentopology]

@HenryL_AI @Recursive_SI Thesis automated AI research and you can use it today! https://t.co/fnAF3UR6aH

[Sergio Charles @eigentopology]

@hallerite @thesis_labs

[Sergio Charles @eigentopology]

@DimitrisPapail @braindersnn This is the future of research. Where do you write your methods and analysis section?

[Sergio Charles @eigentopology]

@eliebakouch When @thesis_labs adds slack

[Sergio Charles @eigentopology]

@gpusteve I prefer manually moving around electrons myself

[Sergio Charles @eigentopology]

@harshbhatt7585 What are your takeaways? They look quite similar upon first inspection

[Sergio Charles @eigentopology]

@ScienceMagazine incredible.

[Sergio Charles @eigentopology]

I'd think bio could get away with less data than language because of stronger priors: SE(3), conservation, geometry, physics. But AF2 -> AF3 dropping IPA kind of went against the idea of inductive biases leading to better sample efficiency. Imo the real difficulty is that biology is multi-scale: atoms -> proteins -> cells -> organisms.

[Sergio Charles @eigentopology]

@rishabh16_ SE(3) is the best Lie group