Thirty five researchers from Stanford, Harvard, MIT, Google, Microsoft, Anthropic, OpenAI, and Amazon just published a paper arguing that nobody actually knows what a "superintelligent" medical AI would look like, because the benchmarks we use to test medical AI are not built to answer that question.
The paper, led by Stanford's Ethan Goh and published in Nature Medicine, argues that current medical AI evaluations reward fluency and test taking performance, not the kind of judgment a clinician actually needs. The authors call for a rigorous, task based framework for measuring medical AI capability, built around real clinical tasks rather than multiple choice exams.
I see the gap this paper is describing every week. I teach physicians and trainees how to evaluate AI tools through MeDucation AI, and the honest answer to "is this tool any good" is usually "compared to what, on which task, judged by whom." Most clinicians are left choosing between blind trust and blanket dismissal, because nobody handed them a real way to test the thing in front of them.
A field that cannot agree on what to measure will keep having the same argument, hype versus skepticism, without either side being able to prove their case.
What would you actually want to see, on a real patient case, before you called a medical AI system competent, let alone superintelligent?
https://t.co/mgnjiH3zr4
Wisdom from Ramon Cajal: For December 7th, let us draw inspiration from Cajal’s reflections on perseverance and self-belief in the face of adversity. In his “Advice for a Young Investigator,” he wrote:
“Deficiencies of innate ability may be compensated for through persistent hard work and concentration. One might say that work substitutes for talent or, better yet, that it creates talent.”
This advice encourages us to embrace challenges with determination. Whether or not we feel naturally gifted, diligence and focus can transform our potential. Cajal reminds us that effort and learning are within everyone’s grasp and can lead to meaningful contributions in any field of inquiry.
https://t.co/exujay7jBm
Muy feliz de haber participado en el Global Summit 2024 @GSIMedicine
Excelentes ponencias de Electrolitos en el perioperatorio y deporte por parte de los doctores Diego Carrillo y @AdrianSotoMota#GlobalSummit#Medicine#POCUS
Excited to share the technical report for RadRotator, our latest #GenerativeAI tool, which enables the rotation of radiographs in 3D space💫
✅Technical Report:
🔗https://t.co/OCEztXg0Nh
✅Website:
🔗https://t.co/X1j3kYVmUE
✅Online Demo:
🔗https://t.co/ygSKkYLOlD
(1/3)
"¡La diferencia entre la felicidad y la desdicha!
ambas son necesarias y útiles, y la muerte o la
desapariciones tan relativo, y la vida también." -Vincent van Gogh
#cartasvg