Puneesh Deora

PhD student at UBC. Working on theory of Deep Learning.

[Puneesh Deora @puneeshdeora]

Check out our recent work on understanding and comparing generalization of Shampoo/Muon with GD 👇

[Bhavya Vasudeva @bhavya_vasudeva]

📢 Late post

Our recent work studies when and why spectrum-aware optimizers like Muon generalize better than standard Euclidean gradient descent (GD)

🧵below (1/N)

[Bhavya Vasudeva @bhavya_vasudeva]

📢 Excited to announce the 2nd edition of MOSS workshop at @COLM_conf. We welcome work that highlights the potential and limits of small-scale in the era of large-scale ML.

CfP: https://t.co/qi8t8xMiJH
Deadline: June 30, 2026

Please help spread the word and see you in SF!

[Daniel Litt @littmath]

My guess is that eventually AI has advantage over us in all aspects of mathematical research. But I think there are two possible worlds where this doesn't happen soonish:

(1) AI exceeds our capabilities in all measurable capabilities, but there are still some (very intangible) intangibles where humans have higher capabilities. In this world we see AI making huge numbers of amazing discoveries but for some weird reason humans sometimes also do so. This could happen if there are some skills that are useful for math research but very hard to measure or train for. I think this is unlikely but possible.

(2) Rather than AI exceeding humans in all capabilities, AI becomes more like *one* (or several) amazing mathematician(s). By which I mean, very technically strong but with a certain point of view, which makes them more likely to make certain discoveries than others. In this world humans are useful because of their greater cognitive diversity--they sometimes think of approaches/questions/ideas that don't occur to the AI. This happens now--there are mathematicians much stronger than I am but I'm still useful (plausibly for reasons other than comparative advantage) because I have some different points of view. I think this is closer to what the near-term trend looks like, but the medium-term is less clear.

In the (IMO more likely, in the medium-to-long term) world where AI does have absolute advantage over us in all aspects, I don't think research ends. Plausibly we have AI producing huge numbers of papers, asking all sorts of interesting questions, and so on. Probably these questions do not include all questions we might ask, so there's still some role where we ask questions (even if the models are better at this, me still might care about questions they haven't asked)!

And IMO the goal of science isn't just "papers," it also includes things like "understanding." Maybe the models have this in their weights or something, but I also want to understand stuff. So we have to extract this understanding from papers or model weights or w/ever. This might look more like hermeneutics than research currently does, but my experience is that trying to understand mathematics that someone else has produced is not actually so different from research. You end up doing a lot of work, developing your own intuition, and so on.

[Puneesh Deora @puneeshdeora]

Now that the hype has settled a little, why is the immediate reaction to dunk on math people the second an AI helps prove a result? Why are some of you so desperate to laugh at mathematicians, what did even they do to you?

[Talia Ringer 🕊🪬 @TaliaRinger]

I suspect for the next few years, AI for math will be most useful for pushing back against our biases that (1) certain false conjectures are true, and (2) certain approachable problems are intractably hard. Mostly by trying things our judgment is now too clouded to try

[Puneesh Deora @puneeshdeora]

I totally get the speeding up new findings angle with AI in Math, but the process of discovery using the traditional way we've known for a while is very rewarding in itself. It is now being killed at a very fast pace and we're not talking much about it.

[Puneesh Deora @puneeshdeora]

Claude be like

[Anthropic @AnthropicAI]

In fact, NLAs suggest Claude suspects it’s being tested across many of our evaluations, even when it doesn’t verbalize its suspicions.

[Puneesh Deora @puneeshdeora]

@PontiEdoardo @icmlconf Been saying for a while that ACs need to be held more accountable. Arguably the most important role in the current system. Revs have some consequences for not doing their job properly, it's time ACs have some too.

[Puneesh Deora @puneeshdeora]

@icmlconf "lowk" "y'all" words I never expected icml account to use. Also, no caps at the start of sentences, missing periods, tough.

[Kimon Fountoulakis @kfountou]

I use GPT-5.5 Pro daily, and I have been using GPT models daily since 5.2. It does pose questions that seem like they could lead to a paper, but then I spend multiple days on these ideas and they flop. The good ideas come from humans. The models don’t have good taste yet.

[Bhavya Vasudeva @bhavya_vasudeva]

I'll be presenting our recent work on how transformers learn to do contextual recall @scifordl workshop tomorrow poster session 1, 11:45 am

[Puneesh Deora @puneeshdeora]

I'm at #ICLR2026 and will be presenting our recent work on in-context learning analogue of benign overfitting @scifordl workshop tomorrow poster session 1, 11:45 am.

[Bhavya Vasudeva @bhavya_vasudeva]

2⃣ On when and why Muon generalizes better than (S)GD
🕙 Poster session 6, 3:15pm-5:45pm
📍Pavilion 3 #809
https://t.co/ZV406xSX8w

[Bhavya Vasudeva @bhavya_vasudeva]

I'm at #ICLR2026 and will present two posters on Day 3:

1⃣ On how Transformer-based language models disentangle and compose latent concepts for in-context learning
🕙 Poster session 5, 10:30am-12pm
📍Pavilion 4 #4008

[ICML Conference @icmlconf]

Announcing the #ICML2026 tutorials!

All ten tutorials will be presented the first day of the conference, Monday July 6.

Read the blog post for more details on the selection process!

[Juno KIM @junokim_ai]

Excited to share our new paper on sharp capacity scaling of the Muon optimizer! Joint work with @EshaanNichani Denny Wu @albertobietti @jasondeanlee:
https://t.co/v1k1B4mSkG (1/7)

[Puneesh Deora @puneeshdeora]

@f14bertolotti I was looking for a wall clock time comparison instead of just steps

[Puneesh Deora @puneeshdeora]

A big part is learning things, learning how to think, what to think, developing taste. Sending papers in the void is no good, but still formalizing and writing things helps. I'm not sure agent-written code would help with these

[Jon Barron @jon_barron]

If I was a grad student today, I would: 1) Not write papers, 2) push my (agent-written) code to a public repo ~weekly, 3) maintain (via agents) a writeup.tex (manually verified) and a skill.md in the repo, and 4) work towards establishing skill usage as the new "citation" format.

[Edgar Dobriban @EdgarDobriban]

AI is getting great at math, but how good is it at solving real research problems in areas outside of those covered by Erdős problems? Towards gauging this, I have started putting together a list of unsolved research problems in mathematical statistics and machine learning, sourced from recent papers in a leading statistics journal, the Annals of Statistics (with some bonus COLT open problems: https://t.co/aT9kyozYsv.

Currently >100 problems.

In my view, much of the value of AI for researchers in the mathematical sciences stems from helping with their own research problems. These are problems without known solutions. There are many math benchmarks, but few with the following properties:

(1) of a realistic research-level, so that solving them can potentially lead to a publication in a top journal (problems discussed in papers already, not contest math, not Millenium problems, not problems created for a benchmark, not problems that have a known solution);
I'd say Erdős problems are the best example of this.

(2) cover problems outside of the usual focus (combinatorics, number theory, ... ) of Erdős problems. Especially under-represented are domains of applied math, along with statistics, operations research, etc.
I'm interested in statistics and ML, so that's where I started, but this could grow over time.

Hope this can grow into something useful to the community! Happy to hear your thoughts...

[Puneesh Deora @puneeshdeora]

I can finally pack the 8D oranges in my kitchen I’ve been putting off for a while. Feeling quite relieved and satisfied.

[Math, Inc. @mathematics_inc]

We are pleased to share that using Gauss, we have completed a ~200K LOC formalization of Maryna Viazovska’s 2022 Fields Medal theorems on optimal sphere packing in dimensions 8 and 24.

This is the only Fields Medal-winning result from this century to be completely formalized, and is the largest single-purpose Lean formalization in history.

We are honored to have assisted @SidharthHarihar1 and the rest of the sphere packing team in this achievement.

https://t.co/DhGDQzLkpH

[Puneesh Deora @puneeshdeora]

@miniapeur Some of the issues I think we could also fix ourselves; I've seen this lot in papers citing other papers
https://t.co/bN32pv1VFr

[Puneesh Deora @puneeshdeora]

Idk if people know this, but google scholar does not index the second reference format type (first it does), and the second is the bibtex you get from arxiv.