AI4Math research focuses on Lean autoformalization and theorem proving. But how do we come up with new conjectures?
In our work with @fin_presented, we study how well can LLM agents use computation to create hypotheses and solve research-level mathematical problems?
arXiv: https://t.co/9xeEVSyFkW
We evaluate 15 LLMs in zero-shot and SageMath-augmented agentic setup on RealMath 133 problems extracted from arXiv mathematical papers. Key findings:
- Tool access improves every model, by 9.7 pp on average. Open-weight models gain 15.3 pp, compared with 6.5 pp for closed frontier models, narrowing the gap between them.
- Most intriguingly, a CAS-augmented agent reproduced a computational mathematician’s workflow: computing intermediate objects, finding patterns, forming conjectures, recovering from errors, and validating formulas across parameters (see the details in the case study).
- Gains vary: #Qwen 3.7-Max rises from 42.1% to 69.9%, a gain of 27.8 pp that brings it close to frontier performance. #Kimi 2.7 gains only 1.5 pp.
- Tool-use behavior is strongly bimodal. Strong agents usually finish in 3-4 tool turns, while weaker agents often exhaust all tool budgets.
- The largest gains are in combinatorics (+18.7 pp) and rings and algebras (+10.7 pp), while algebraic topology and group theory remain difficult.
- Recovery after a failed tool call ranges from 16% (#Sonnet-5) to 77% (GPT-5.5) across models. The ability to revise a strategy after receiving computational feedback separates effective agents more clearly than the raw number of errors.
Interesting observations about some models:
- #GPT 5.5 leads in both solve rate and efficiency, reaching a 75.2% accuracy with the lowest token usage among tool-enabled agents.
- #MiniMax M3 is the least efficient, using the most tokens per problem and achieving substantially lower accuracy.
- #Opus 4.8 exceeds Opus 4.7 by only one solved problem.
- #Grok 4.3 shows one of the worst results and produces 248/336 SyntaxErrors🥲.
- #Fugu Ultra shows the smallest increase in token usage with tool access, at 4.5×, averaging 70k tokens per problem.
Today we are presenting our poster at the @ai4mathworkshop at @icmlconf. Come by to discuss our work.
#ICML2026 #AI4Math #AI #Agentic #LLM #Mathematics
Our LLM Applied Scientist German Magai @MetatrolN presents at the 3rd AI For Math Workshop at ICML 2026 (July 11, Seoul): "Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics" with Pavel Snopov @fin_presented (UT Rio Grande Valley).
Give frontier models a computer algebra system and watch them work like mathematicians: guess, compute, break something, fix it, check the pattern, commit. The size of the payoff had little to do with model strength. What set the winners apart was recovering after a failed call, not raw reasoning.
If you're at ICML 2026, find German on the 11th to chat about agentic reasoning, math, and why more tokens don’t automatically mean more right answers.
We curated 40+ quantum mechanics datasets covering 250+ quantum methods & 1.5B geometries. All of this is easily accessible through the OpenQDC library. 🧵
Harness the power of QM data with a single line of code: https://t.co/4nurV2Fwst
Blog: https://t.co/R1I1FK2Z0x
What if any neural network could go topological? While being just as (or more) general, equivariant, and expressive as other Topological Deep Learning models? Without a hacky, case-by-case software struggle?
I am beyond excited to introduce TopoTune💫
https://t.co/7XEVizGc5q
What breakthroughs can be expected from #AI? The new founding director of the #AITHYRA Institute @mmbronstein discusses the future of AI in a public lecture at the Austrian Academy of Sciences. Watch the full lecture here ➡️ #boehringeringelheimstiftung https://t.co/e6o62VpJmO
@short_cast@cloneofsimo It seems to me that it is important to remain on the embedded manifold. But if you take gradient steps in the ambient space, you risk moving away from it, don't you?
It’s been a week since we wrapped up an exciting school on topology, combinatorics, and data analysis in Voronezh, coorganized with my friends @bekemax, Andrey Ryabichev, and Dima Vasiliev. Tnx to CS HSE and AMM VSU for the support!
More info (in Russian): https://t.co/adWA81N29o
Deep Learning on Graphs: Past, Present, And Future
Graph representation learning has recently become one of the hottest topics in machine learning.
One particular instance, graph neural networks, is being used in a broad spectrum of applications ranging from 3D computer vision and graphics to high energy physics and drug design.
Despite the promise and a series of success stories of graph deep learning methods, we have not witnessed so far anything close to the smashing success convolutional networks have had in computer vision.
In this talk, Michael Bronstein outlines his views on the possible reasons and how the field could progress in the next few years.
Listen to this episode for a succinct introduction to GNNs, and find for yourselves how Michael Bronstein’s predictions and points from Knowledge Connexions 2020 holding up in 2024.
–-
@mmbronstein is a professor at Imperial College London, where he holds the Chair in Machine Learning and Pattern Recognition, and Head of Graph Learning Research at Twitter.
He also heads ML research in Project CETI, a TED Audacious Prize-winning collaboration aimed at understanding the communication of sperm whales.
---
To learn more about Graph AI, Knowledge Graphs, and LLMs and meet Leaders and Innovators join us in Connected Data London on December 11-13. We've been working with Knowledge Graphs for more than 20 years, and sharing it with the world since 2016.
#KnowledgeGraph #ConnectedData #datascience #podcast #KnowCon2020 #GNN #DeepLearning #MachineLearning #EmergingTech #AI
https://t.co/PXBmXLIvgw
Had a great time participating in the @GRaM_org_ challenge at #ICML2024! It was an incredible experience implementing different liftings and passing constructions from pure math. Many thanks to the organizers @PyT_Team_@gbg1441@lev_telyatnikov!
Topological Deep Learning is an immensely powerful and fast emerging field. Our new literature review https://t.co/MLp4AaV8vf is out and here’s why I’m very excited about it🧵1/5
Has anyone ever come across this way of visualizing the convolution of two functions f and g: Look at the graph of the two-variable function f(x)g(y), and consider diagonal slices over the line x + y = k.
The area of those slices represents (f * g)(k).
I wanted to score 100 points, but I was just a little short. I scored the least in Speaking; perhaps my laptop's built-in microphone was what hindered me from hitting that perfect 100. Or maybe not 🤨