Top Tweets for #AutoMedBench

🩺 Can AI agents conduct medical research end-to-end, just like human researchers? Introducing AutoMedBench — the first workflow-aware benchmark for medical AutoResearch agents. 🧪 📄 Paper: https://t.co/2H666zmPov 🌐 Project: https://t.co/bYKEVcfqEU 💻 Code: https://t.co/z37BfBBNm5 Medical agents are rapidly evolving from answering questions to conducting end-to-end medical-AI research: loading datasets, building pipelines, debugging failures, running inference, and submitting results. Yet most benchmarks only evaluate the final answer. A good score can hide a broken process. A failed run often reveals nothing about where the agent went wrong. 🔬 AutoMedBench evaluates the entire workflow. Every run is decomposed into 5 stages: 📝 Plan → ⚙️ Setup → 🔍 Validate → 🚀 Inference → 📦 Submit Instead of a single score, AutoMedBench diagnoses which stage succeeds or fails—and why. 📊 Benchmark scope • 5 medical AI tracks: Segmentation, Enhancement, VQA, Report Generation, and Lesion Detection • Lite & Standard tiers (same data/metrics, different scaffolding) • Long-horizon tasks averaging ~33 agent turns • Full logs of actions, tokens, runtime, cost, and error codes We put today's frontier agents to the test: 🏆 Overall leaderboard 🥇 #Opus 4.6 — 66.5 🥈 #GLM-5 — 61.6 🥉 #Gemini 3.1 Pro — 59.0 4️⃣ #ChatGPT-5.4 — 55.3 5️⃣ #MiniMax-M2.5 — 51.6 6️⃣ #Qwen3.5 — 51.2 But no single model dominates everything: GLM-5 leads VQA, while Opus 4.6 leads most other tracks. ⚠️ The key finding Across thousands of runs, Validate is the weakest stage while Setup is the strongest. Today's agents are much better at making a pipeline run than ensuring it is correct before large-scale inference. 📉 The bottleneck isn't medical knowledge 🔍 Verification & recovery errors: 37.7% 📦 Deliverable & submission errors: 38.1% 🧠 Task-understanding errors: only 0.9% Even a single fired error code reduces the overall score by 48%. The next frontier for medical AI agents is not more knowledge—it's workflow reliability, verification, and self-correction. #AutoResearch #MedicalAI #AIAgents #HealthcareAI #AgenticAI