Marvin Li

Harvard '25 | Building theory for generative models

[Marvin Li @marvin_li03]

New paper alert!🚨 What do LLM reasoning, diffusions, & jailbreaks have in common? 🤔

All exhibit critical windows📈--a sudden formation of distinctive features during sampling, e.g. correctness or toxicity.

We present a unifying theory of critical windows for diffusion & LLMs.

[Daniel Rosiak @DanRosiak]

Over the past few months, amid wave after wave of impressive ai-math results, a bizarre framing has become increasingly common on this site: a mathematician's reaction to all this supposedly reduces to "truth vs. ego".

Apparently either you are delighted by every ai breakthrough because you care about truth, or you never really loved math, you only loved being the person who solves things (ego).

There's a lot going on here, and many caveats (like i do not actually think it's as dire as people suggest), but I'll skip all that to focus on attacking the framing.

I find this framing ridiculous and disingenuous.

Suppose a device has been invented that can make a child fully literate in 60 seconds. Decoding words, but also appreciating literature, understanding things like metaphor, and digesting hard books.

As a parent, you're probably initially delighted that the child can read. You can't wait to talk to them about so many things and witness how their mind works! Incredible!

But it would be astounding (and frankly concerning) if you felt no loss here at all, if you did not also mourn: mourn all the evenings you had imagined spending sounding out words together, watching comprehension gradually appear, helping them discover their favorite books, returning to those together.

If someone interpreted that aspect of mourning as equaling:

"You wanted your child to remain ignorant so that you could feel important!"

I would assume that there was something very off with the person suggesting that.

The much more likely train of thought here is:

"Something extremely valuable (to me, to my child, and to humanity) was contained in the shared process by which this sort of knowledge was acquired."

Or suppose someone installs a teleporter at the base of Everest. Anyone can now reach the summit safely in five seconds! This is astounding and really good for people who want the view. People claim it "democratizes" access to summit and prevents deaths. Hard to argue with that.

It would be ridiculous to then tell old-school climbers:

"You claim to love reaching summits, yet you seem unhappy that everyone can now reach one much more quickly. Evidently climbing was only about ego and exclusivity!"

The obvious reply would be:

"There has been a serious misunderstanding. My love of climbing was never about valuing occupying the coordinates at the top. The ascent was a core part of it: the preparation, endurance, testing courage, overcoming fear, pushing the limits of judgment, companionship, failure, and transformation through difficulty. These were always the main things. A teleport may be a better summit-reaching technology, but it can hardly be called a better form of climbing."

Likewise, math colleagues grieving seem to be saying: a theorem machine may be a better theorem-reaching technology without being a better form of doing mathematics. The sadness they are expressing seems to come from the threat of losing an activity that organized attention, made time meaningful, and has been one of the most impactful forms of life in human history.

Those calling this grief "ego" seem to want to imply that math was only ever a circuitous method for increasing the global inventory of theorems, and that any of its form of life aspects were secondary to this.

But...YOU ARE A HUMAN BEING. EVERYTHING YOU DO, EVERYTHING YOU ARE, IS SOCIAL. EVERYTHING.

So ask yourself:
Is math more like removing tumors, where the process is an unfortunate means to the result? Or is it more like climbing, cooking, teaching, etc., one of so many activities in which the process is a large part of the good?

Of course ego exists in math, as it does in every single human vocation. But it does not follow that every grief about what is happening is grief over lost superiority or specialness.

You can celebrate the production of more theorems while grieving the possible disappearance of everything involved in math as form of life. This shouldn't be controversial

[Scott Armstrong @scottnarmstrong]

my twitter feed is now just AI for math, and there are essentially just two types of comments: (A) those gleefully unleashing pent-up resentment toward mathematics and mathematicians (i.e. written by ppl who did not enjoy undergraduate analysis, example @Noahpinion ) and (B) mathematicians who have their entire self worth and reason for existing tied to their cleverness, who are now in despair.

hey everybody we're just a bunch of stupid apes trying to comprehend the universe. it is cool if we can build machines to accelerate our understanding. math is not about you, me, or our egos. nobody needs to jump out the window, chill out

[Grace Li @grx_xce]

Today, we're introducing @Intelligence_ai.

In 6 months, as a team of 10, we scaled from $5M to $60M ARR and 5.5M users across 190+ countries.

We raised a $7.9M seed, led by @IndexVentures with participation from @conviction, @A_StarVC, and @combinator to build DesignArena, a universal interface for accessing and evaluating the world's AI capabilities.

Most evaluations try to simulate the real-world. We believe the real-world is the ultimate verifier.

People come to @DesignArena with a request. Models compete to fulfill the request, and users determine what works best for them.

Their live user behavior evaluates the models, improves how work is routed, and helps people access the right intelligence.

We've helped the world's leading frontier labs break the news on their SOTA capabilities.

What's the limit? Join us and find out.

[Ahmad Beirami @abeirami]

Mandatory Conjecture post!

2 years ago, we conjectured a tighter upper bound than log(n) - (n-1)/n on the KL divergence of best-of-n with the reference model (Conjecture 4.4 in our ICML'25 paper). @th33rtha and I thought about it on and off for 2 weeks and did not resolve it.

Today, I gave it to Fable 5 and GPT-5.6 Sol over lunch. Both came back with a legible proof within minutes!

[alz @alz_zyd_]

some people seem to have this absurd view that "all science is applied math, if we solve math we solve science"

This is a completely ridiculous view. It has no support in the history of science, or common sense

[Alex Meterez @alexmeterez]

1/8 How well does the simplest model in optimization theory, the quadratic, actually describe LLM pretraining? Surprisingly well. We linearize a 150M model at checkpoints throughout training and the local Taylor expansion tracks the true loss for up to 10% of the training budget.

[Denny’s @DennysDiner]

@JensenHuang @nvidia

[alz @alz_zyd_]

it's been a few years now and nobody has vibecoded anything worth anything

this shows that code has never been the bottleneck for building anything worth building

[Marvin Li @marvin_li03]

@lucyjcai @clairemao78 @tryinstance Congrats!

[Aryaman Arora @aryaman2020]

@NepalAadim https://t.co/rQXmeAybzJ
https://t.co/CrDGrKELC9
https://t.co/kFWPZjGtIK

[Apoorva Panidapu @apoorvapanidapu]

Earlier this month I volunteered at Stanford’s Future of Math symposium, and ever since, I've been puzzling through what it now means to pursue mathematics as a student in the age of AI. I wrote an essay to make sense of it all: https://t.co/cYyUDqSady

[François Chollet @fchollet]

The jobs of the future will require high adaptability and creativity, focusing on complex problem framing rather than repetitive execution or specialized skills

[Michael Y. Li @michaelyli_]

You're wasting FLOPs when scaling inference compute: by independently sampling parallel attempts, you burn compute rediscovering the same solutions.

Introducing QuasiMoTTo: we scale parallel sampling with correlated samples instead! These samples have higher coverage, are marginally exact draws from the LLM, and can be generated in parallel.

Result: same performance with 25-47% fewer samples in test-time scaling + 50% fewer training steps in RL!

In our new paper, we explore the design space of correlated samplers. Work with co-authors @probablynotaz9 (co-lead), @gandhikanishk, @noahdgoodman, and Emily Fox!

[Sarvesh Patil @servo97]

Interaction with the real world is the major bottleneck in robot learning. So what would robot RL look like if we didn’t need to limit compute per interaction? Our latest work, Off-Policy Generative Policy Optimization (OGPO, accepted to ICML26) embarks on answering this question (spoiler alert: when done correctly, it helps massively!).
🧵(1/N)

[François Chollet @fchollet]

Only Art and Science raise humanity to the sublime. But the magic isn't in the output. It's in the process of creation and discovery, and the experience of feeling and understanding.

[François Chollet @fchollet]

The true measure of a software engineer isn't their ability to write clever code. It's their ability to ruthlessly protect the codebase from unnecessary cleverness.

[Mayor Zohran Kwame Mamdani @NYCMayor]

GOOD MORNING NEW YORK CITY.

[Jaeyeon (Jay) Kim @Jaeyeon_Kim_0]

We’re organizing the first workshop on non-AR Language Models (eg discrete diffusion models)! Excited to be part of this and to collaborate with so many good researchers / and have amazing speaker line-up!!

Check it out here: https://t.co/YUIiC70Lpb

[Pope Leo XIV @Pontifex]

The Holy Spirit challenges us today regarding our relationship with technology and the ongoing digital revolution. Technology has the power to heal, connect, educate and protect our common home; but it can also divide, exclude and generate new forms of injustice. #MagnificaHumanitas

[Pope Leo XIV @Pontifex]

In the era of #ArtificialIntelligence, when human dignity is threatened by new forms of dehumanization, ours is the pressing duty to remain profoundly human. We must lovingly safeguard the grandeur of humanity bestowed upon us and revealed in its fullness in Christ, the splendor of which no machine can ever replace. #MagnificaHumanitas
https://t.co/6i9MWs6LJl