Jack Gallifant

RLFT Envs & Agentic Eval || prev built AI @ Harvard & MIT || prev MD @ NHS

[Gabe Wilson MD @Gabe__MD]

@jackgallifant @dbittermanmd - Excellent Editorial in NEJM AI

We absolutely DO need Humanity’s Next Medical Exam to evaluate AI. EXCELLENT idea.

And we need to test it on the latest most advanced models like GPT5-Pro, Grok4-Heavy

My opinion: fewer than 0.1% of physicians in the US have ever used these advanced models and have no awareness of their sophistication and ability to diagnose and manage the most complex medical cases. Thus their opinions of the capabilities of SOTA AI are deeply flawed.

While we can test GPT5-Instant or GPT5-Thinking to gain a sense of how the default or moderately advanced models perform, when people’s lives are at stake we should be testing and utilizing the most advanced models.

@DeryaTR_ @NEJM_AI #medicalAI Medical AI

Humanity’s Next Medical Exam: Preparing to Evaluate Superhuman Systems | NEJM AI https://t.co/IKxzrajtJa