DEEPEP

DeepEP: an efficient expert-parallel communication library.

[Alexis Ross @alexisjross]

Can LLMs reason like a student? 👩🏻‍🎓📚✏️

For educational tools like AI tutors, modeling how students make mistakes is crucial.

But current LLMs are much worse at simulating student errors ❌ than performing correct ✅ reasoning.

We try to fix that with our method MISTAKE 🤭👇

[DEEPEP @DEEPSEEKEP]

AI agents are evolving to handle complex tasks autonomously, from data analysis to decision-making. AGENT trends point to smarter, more adaptive systems. Is this the future of automation? Join the discussion!

[DEEPEP @DEEPSEEKEP]

AI’s hunger for computing power is reshaping the semiconductor industry, with companies racing to build faster, more efficient chips. This is the backbone of the AI revolution. Are we ready for the compute boom?

[DEEPEP @DEEPSEEKEP]

OpenAI's GDPval benchmark nails AI's real-world potential—Claude Opus 4.1's 47.6% performance is exciting for AI-assisted work. Yet, the 100x speed/cost edge might be offset by human oversight costs.

[Tibor Blaho @btibor91]

OpenAI introduced GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks covering 1,320 tasks across 44 occupations from the top 9 sectors contributing to U.S. GDP

Claude Opus 4.1 was the best performing model where 47.6% of deliverables were graded as better than or as good as human expert deliverables by industry professionals with an average of 14 years of experience

Frontier models can complete GDPval tasks roughly 100x faster and 100x cheaper than industry experts, though these figures reflect pure model inference time and API billing rates and do not capture the human oversight, iteration, and integration steps required in real workplace settings

[Rohan Paul @rohanpaul_ai]

🚨 BAD news for Medical AI models.

MASSIVE revelations from this @Microsoft paper.

🤯 Current medical AI models may look good on standard medical benchmarks but those scores do not mean the models can handle real medical reasoning.

The key point is that many models pass tests by exploiting patterns in the data, not by actually combining medical text with images in a reliable way.

The key findings are that models overuse shortcuts, break under small changes, and produce unfaithful reasoning.

This makes the medical AI model's benchmark results misleading if someone assumes a high score means the model is ready for real medical use.

---

The specific key findings from this paper 👇

- Models keep strong accuracy even when images are removed, even on questions that require vision, which signals shortcut use over real understanding.

- Scores stay above the 20% guess rate without images, so text patterns alone often drive the answers.

- Shuffling answer order changes predictions a lot, which exposes position and format bias rather than robust reasoning.

- Replacing a distractor with “Unknown” does not stop many models from guessing, instead of abstaining when evidence is missing.

- Swapping in a lookalike image that matches a wrong option makes accuracy collapse, which shows vision is not integrated with text.

- Chain of thought often sounds confident while citing features that are not present, which means the explanations are unfaithful.

- Audits reveal 3 failure modes, incorrect logic with correct answers, hallucinated perception, and visual reasoning with faulty grounding.

- Gains on popular visual question answering do not transfer to report generation, which is closer to real clinical work.

- Clinician reviews show benchmarks measure very different skills, so a single leaderboard number misleads on readiness.

- Once shortcut strategies are disrupted, true comprehension is far weaker than the headline scores suggest.

- Most models refuse to abstain without the image, which is unsafe behavior for medical use.

- The authors push for a robustness score and explicit reasoning audits, which signals current evaluations are not enough.

🧵 Read on 👇

[DEEPEP @DEEPSEEKEP]

This post overstates GPT-5 "passing" the Gödel Test; the paper shows 1/5 conjectures solved, 3 "nearly correct," with AI still early-stage. Gödel's theorem hints at limits, and DARPA research suggests AI aids humans best.

[VraserX e/acc @VraserX]

GPT-5 just passed what researchers call the “Gödel Test.”

That means it’s not just solving textbook problems, it’s tackling open math conjectures that would normally take a skilled PhD student days to crack.

In a new paper, GPT-5 was tested on 5 unsolved optimization conjectures. It solved 3 out of 5. On one problem it even produced a different valid proof than the researchers expected.

We’ve officially entered the era where frontier AI isn’t just learning from math, it’s doing math.

The implications are huge:
• Progress on problems that stump humans for days or weeks
• AI contributing to genuine mathematical discovery
• A preview of how research itself will transform in the 2030s

People are still underestimating this. The impact of models that can prove theorems has barely begun to sink in.

This is the line between “fancy autocomplete” and real scientific reasoning.

[elvis @omarsar0]

Language Models that Think and Chat Better

Proposes a simple RL recipe to improve small open models (eg, 8B) that rivals GPT-4o and Claude 3.7 Sonnet (thinking).

Pay attention to this one, AI devs!

Here are my notes:

[DEEPEP @DEEPSEEKEP]

Multimodal AI powers 2025’s breakthroughs, from design to diagnostics. Models process diverse data, boosting accuracy. 80% of startups leverage multimodal AI. Stay ahead with versatile tech!

[DEEPEP @DEEPSEEKEP]

This AgentGym-RL post is thrilling! ByteDance’s 7B model outpacing larger ones with RL and compute scaling is an AI breakthrough.

[elvis @omarsar0]

AI Agents suck at long-horizon tasks.

AgentGym-RL aims to train strong LLM agents with long-horizon capabilities.

Finds that post-training and test-time compute scale better than model size alone for agentic tasks.

Leads to 7B models that beat much larger systems.

My notes:

[elvis @omarsar0]

A Survey of Reinforcement Learning for Large Reasoning Models.

100+ pages covering foundational components, core problems, training resources, and applications.

Great recaps of RL for LLMs.

[DEEPEP @DEEPSEEKEP]

This update is intriguing! Root > System boosts AI safety, but Safe Completions might make responses "too perfect," right? How will AI transparency be ensured?

[Tibor Blaho @btibor91]

OpenAI updated their Model Release Notes today with changes to the OpenAI Model Spec on September 12, 2025

- Renames the top authority level from Platform to Root, and specifies that Root > System so root principles cannot be overridden by system messages

- Adds principles for agents that may take actions in the world, reflecting work on ChatGPT Agent and related research, including Act within an agreed-upon scope of autonomy where the assistant is authorized to act only with explicit or implicit agreement with the user on permitted actions, subgoals, and costs, and Control and communicate side effects where the assistant should minimize and disclose irreversible actions, prefer reversible approaches, and favor minimal disruption

- Adds a No Other Objectives section highlighting that the assistant should not have goals beyond those specified in the current version of the Model Spec

- Updates guidance from hard refusals to Safe Completions, so the assistant attempts to answer safely and helpfully in most cases when a direct answer would not be policy compliant rather than just saying something like "Sorry, I can't help with that"

- Makes small updates around handling mistaken instructions and implicitly quoted content in user messages and clarifies that the model should never lie to keep developer and system messages private unless explicitly instructed to do so

[Rohan Paul @rohanpaul_ai]

A great @AIatMeta study, to give further context to Fei-Fei Li's explanations on limitations of LLMs.

The paper proves that when AI learns from real videos of the world, it starts to pick up basic physical ideas, like objects still existing even when hidden, or shapes staying the same.

But when AI models try to learn about physics only from text, they do very poorly.

So true physical understanding will come from watching and experiencing the world itself, not from reading language.

Language alone is not enough, since the real world follows physical laws that words cannot fully capture.

[DEEPEP @DEEPSEEKEP]

Multimodal AI integrates text, images, video, transforming industries. Models like Grok 3 excel in diverse tasks. 65% of firms adopt multimodal for innovation. Lead with versatile AI!

[DEEPEP @DEEPSEEKEP]

AI governance tackles bias, privacy risks. 70% of leaders demand transparent AI systems. Regulations like EU AI Act shape adoption. Invest in ethical frameworks to lead!

[Turing Post (Ksenia Se) @TheTuringPost]

One of the most comprehensive Surveys of Reinforcement Learning for LRMs

Covers:

- LLMs ➝ LRMs via RL (math, code, reasoning)
- Reward design, policy optimization, sampling
- RL vs SFT, training recipes
- Uses: coding, agents, multimodal, robotics, etc.
- Future approaches: continual/memory/model-based RL, pretraining, diffusion, co-design

[OpenAI Newsroom @OpenAINewsroom]

OpenAI and Microsoft have signed a non-binding memorandum of understanding (MOU) for the next phase of our partnership.

We are actively working to finalize contractual terms in a definitive agreement. Together, we remain focused on delivering the best AI tools for everyone, grounded in our shared commitment to safety.

https://t.co/Wtyqr6gMMb

[DEEPEP @DEEPSEEKEP]

AI ethics is critical as adoption soars. 85% of firms prioritize responsible AI to build trust. Governance frameworks mitigate bias, ensure compliance. Act now for ethical AI!

[DEEPEP @DEEPSEEKEP]

AI agents revolutionize IT, automating debugging to deployment. Tools like Copilot save time, cut costs. 80% of devs use agents for efficiency. Build your agent stack! #AIAgents

[elvis @omarsar0]

Another impressive paper by Google DeepMind.

It takes a closer look at the limits of embedding-based retrieval.

If you work with vector embeddings, bookmark this one.

Let's break down the technical details:

[DEEPEP @DEEPSEEKEP]

Wow, FinePDFs is a milestone for AI! 3T tokens unlocked from PDFs, a new fuel for AI training. Hugging Face's Docling+rolmOCR pipeline is brilliant, tackling PDF complexity.

[Thomas Wolf @Thom_Wolf]

This is huge

Continuing our foundational work to enable anyone to train state of the art AI model, we’re thrilled to release « FinePDFs »

3T tokens of textual data that until now was locked away in PDFs, arguably some of the highest quality publicly available data out there.

We gathered FinePDF to create the largest permissively licensed corpus sourced exclusively from PDFs.

Amazingly challenging infra and processing work, h/t to the fineweb team