German Magai

PhD in CS, LLM Applied Scientist at Noeon Research, Japan.

[German Magai @MetatrolN]

AI4Math research focuses on Lean autoformalization and theorem proving. But how do we come up with new conjectures?

In our work with @fin_presented, we study how well can LLM agents use computation to create hypotheses and solve research-level mathematical problems?

arXiv: https://t.co/9xeEVSyFkW

We evaluate 15 LLMs in zero-shot and SageMath-augmented agentic setup on RealMath 133 problems extracted from arXiv mathematical papers. Key findings:
- Tool access improves every model, by 9.7 pp on average. Open-weight models gain 15.3 pp, compared with 6.5 pp for closed frontier models, narrowing the gap between them.
- Most intriguingly, a CAS-augmented agent reproduced a computational mathematician’s workflow: computing intermediate objects, finding patterns, forming conjectures, recovering from errors, and validating formulas across parameters (see the details in the case study).
- Gains vary: #Qwen 3.7-Max rises from 42.1% to 69.9%, a gain of 27.8 pp that brings it close to frontier performance. #Kimi 2.7 gains only 1.5 pp.
- Tool-use behavior is strongly bimodal. Strong agents usually finish in 3-4 tool turns, while weaker agents often exhaust all tool budgets.
- The largest gains are in combinatorics (+18.7 pp) and rings and algebras (+10.7 pp), while algebraic topology and group theory remain difficult.
- Recovery after a failed tool call ranges from 16% (#Sonnet-5) to 77% (GPT-5.5) across models. The ability to revise a strategy after receiving computational feedback separates effective agents more clearly than the raw number of errors.

Interesting observations about some models:
- #GPT 5.5 leads in both solve rate and efficiency, reaching a 75.2% accuracy with the lowest token usage among tool-enabled agents.
- #MiniMax M3 is the least efficient, using the most tokens per problem and achieving substantially lower accuracy.
- #Opus 4.8 exceeds Opus 4.7 by only one solved problem.
- #Grok 4.3 shows one of the worst results and produces 248/336 SyntaxErrors🥲.
- #Fugu Ultra shows the smallest increase in token usage with tool access, at 4.5×, averaging 70k tokens per problem.

Today we are presenting our poster at the @ai4mathworkshop at @icmlconf. Come by to discuss our work.

#ICML2026 #AI4Math #AI #Agentic #LLM #Mathematics

[Noeon Research @NoeonAI]

Our LLM Applied Scientist German Magai @MetatrolN presents at the 3rd AI For Math Workshop at ICML 2026 (July 11, Seoul): "Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics" with Pavel Snopov @fin_presented (UT Rio Grande Valley).

Give frontier models a computer algebra system and watch them work like mathematicians: guess, compute, break something, fix it, check the pattern, commit. The size of the payoff had little to do with model strength. What set the winners apart was recovering after a failed call, not raw reasoning.

If you're at ICML 2026, find German on the 11th to chat about agentic reasoning, math, and why more tokens don’t automatically mean more right answers.

[German Magai @MetatrolN]

This Medium post provides a simple and intuitive explanation of the applied aspects of our review paper, "Sheaf Theory: From Deep Geometry to Deep Learning."

https://t.co/hMaRWOQmJ0

[German Magai @MetatrolN]

@ArieBaba11 @grok @AMAZlNGNATURE This is definitely Russian. The man in the video is talking about a mother bear named Masha running around with her cubs. And when he fell, he said: I knew you'd knock me down.

[arXiv math.AT Algebraic Topology @mathATb]

Anton Ayzenberg, Thomas Gebhart, German Magai, Grigory Solomadin: Sheaf theory: from deep geometry to deep learning https://t.co/jOYrtG7L7M https://t.co/QOiBhzsAFp https://t.co/WRZXUtGIqq

[German Magai @MetatrolN]

Media art exhibition "Robot-dog on a chain" of @toodooda. Being near a robo-dog we feel both anxiety, because the dog behaves aggressively, but at the same time we feel some empathy.

I would put this dog at the entrance to a conference about AI Safety.

[German Magai @MetatrolN]

@toodooda @takahiroanno where exactly in Tokyo is this exhibition taking place? I can't find any information anywhere

[German Magai @MetatrolN]

@okajimania where exactly in Tokyo is this exhibition taking place? I can't find any information anywhere

[Sam Altman @sama]

OPENAI ROADMAP UPDATE FOR GPT-4.5 and GPT-5:

We want to do a better job of sharing our intended roadmap, and a much better job simplifying our product offerings.

We want AI to “just work” for you; we realize how complicated our model and product offerings have gotten.

We hate the model picker as much as you do and want to return to magic unified intelligence.

We will next ship GPT-4.5, the model we called Orion internally, as our last non-chain-of-thought model.

After that, a top goal for us is to unify o-series models and GPT-series models by creating systems that can use all our tools, know when to think for a long time or not, and generally be useful for a very wide range of tasks.

In both ChatGPT and our API, we will release GPT-5 as a system that integrates a lot of our technology, including o3. We will no longer ship o3 as a standalone model.

The free tier of ChatGPT will get unlimited chat access to GPT-5 at the standard intelligence setting (!!), subject to abuse thresholds.

Plus subscribers will be able to run GPT-5 at a higher level of intelligence, and Pro subscribers will be able to run GPT-5 at an even higher level of intelligence. These models will incorporate voice, canvas, search, deep research, and more.

[Posts Of Cats @PostsOfCats]

getting his haircut

[Kat ⊷ the Poet Engineer @poetengineer__]

seeing differently

[Leland McInnes @leland_mcinnes]

An ArXiv data map you can browse yourself:

https://t.co/ulpNmynrRf

(an early demo; please be gentle; use shift to lasso-select; click on a point to open the paper)

[Conference on Language Modeling @COLM_conf]

🏆🏆🏆🏆 Four papers at COLM were awarded outstanding paper awards. Congratulations to the authors!

[Sakana AI @SakanaAILabs]

Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!

https://t.co/jC7g5GPVsE

From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.

Here are 4 example Machine Learning research papers generated by The AI Scientist.

We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!

Paper: https://t.co/lTQ8UenFHk
GitHub: https://t.co/Im53whVeAq

Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.

Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.

We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project.

[LOGML Summer School @LogmlSchool]

🎓 And that's a wrap on the LOGML Summer School! 🥰

It was an incredible experience organizing it. A huge thank you to all our speakers for their insightful talks, to our mentors for making it an unforgettable experience for the students, and to all of you for attending! 👏

[GRaM Workshop at ICLR 2026 @GRaM_org_]

The #GRaM panel discussion on scale vs geometric modeling has begun! Great to have @vdbergrianne , @bose_joey , @phillip_isola , @ZKadkhodaie , and @wellingmax weighing in on this timely topic, moderated by @erikjbekkers

[ヤギの人🐐 @yusai00]

【蜂注意】ミツバチの天敵であるスズメバチに対する、養蜂家の恨みつらみが詰まったハチミツがもう狂気の沙汰。「命をかけて」生け捕りにしたオオスズメバチを、生きたままハチミツに漬けこむという斜め上のサイコっぷりで、買わずにはいれなかった。

[German Magai @MetatrolN]

What about detecting AI-generated images? See the preprint "Improving Interpretability and Robustness for the Detection of AI-Generated Images" https://t.co/tOdm83QYgU

#aisafety

[German Magai @MetatrolN]

Can LLM (GPT) model complex algebraic structures? Apparently, yes. Our "Applying language models to algebraic topology: generating simplicial cycles using multi-labeling in Wu's formula" paper was accepted to #ICML2024 . See you at the conference.

https://t.co/DHtsjaq57o

[German Magai @MetatrolN]

@overleaf when I decided to sit down to write an article over the weekend...