Any new field that comes on the scene quickly can vanish just as quickly. Witness data science, personalized medicine, precision medicine, many 'omics, and hopefully digital twins.
🚨 Medical AI has an evidence problem.
Not a performance problem.
An evidence problem.
A brilliant Nature Medicine editorial says what many people avoid saying:
Stop showing us AUROC.
Show us clinical value.
Because:
- sensitivity
- specificity
- calibration
- benchmark performance
…do not prove better patient care.
A model can be technically excellent and still be:
❌ clinically useless
❌ workflow-disruptive
❌ dangerously misleading
If:
- outputs arrive too late
- nobody acts on them
- clinicians can’t trust them
- they simply automate bad decisions
then the AI has zero medical value.
Even with a beautiful ROC curve.
This is the real problem with many “medical copilots” and guideline-centric tools like OpenEvidence:
- They optimize retrieval of what is already written.
- Not understanding of what is actually happening.
- They replicate consensus.
- Not reasoning.
- Medicine is not benchmark logic.
- It is uncertainty management.
The strongest line in the editorial:
“The stronger the claim, the stronger the evidence needed.”
Exactly.
If you claim:
👉 “Our AI improves outcomes”
you should need more than retrospective validation and polished dashboards.
Otherwise, that’s not innovation.
That’s marketing.
My provocative take:
We are not automating intelligence.
We are often automating:
- bad workflows
- rigid guideline thinking
- and weak clinical reasoning
…at scale.
Which means:
we are industrializing mediocrity.
The future of medical AI is not better models.
It is better proof.
Until then:
AI risks becoming beautifully engineered overconfidence.
Prediction: we’re about to enter a period of many negative “individualized treatment strategy” RCTs, because:
1. The need for “individualized treatment” is an appealing idea, so we want to believe its true in excess of the evidence supporting that it actually is true
2. Existing evidence of HTE is generally weak, because it usually misattributes the sources of variation (see citation below)
3. Even when there is latent HTE, our theories that guide individualized strategies are rudimentary and unreliable. We choose a rule so simplistic - so as to be operationalizable - that it has little chance of actually selecting people with atypical responses.
Interesting, nuanced, and sometimes contentious discussion about the validity and interpretation of confidence intervals for observational research: https://t.co/VUOcYkNNz7 #epidemiology#Statistics
The author seems to be making the age-old error of using quantile groups to understand risk patterns. If the type of food consumed matters it will matter on an individual dose-response basis, not according to how many people are like you. https://t.co/prx3A44MTN
1/5 How reproducible are phase III oncology trials?
📊 With @AlexSherryMD@ErikVanZwet@ebludmir et al. We analyzed 632 RCTs (~496,000 pts). The data offer both reassurance and caution for clinicians and researchers. https://t.co/SgrRgv38UI 👇#MsaouelLab
An elastic net (lasso + ridge regression) approach applied to the win ratio for variable selection and risk prediction for hierarchical composite outcomes.
Check https://t.co/nbxTdxodSz for R tools and demonstrations.
Paper accessible at https://t.co/zDUYvTZyQx.
Tutorials in Biostatistics
Guidelines and Best Practices for the Use of Targeted Maximum Likelihood and Machine Learning When Estimating Causal Effects of Exposures on Time‐To‐Event Outcomes. Talbot et al. Statistics in Medicine. https://t.co/kVYJNWVwsf
You’ve been told to fear:
Egg yolks
Ghee / Butter
Paneer
Red meat
But never questioned:
Pizza
Kachori, samosa, vada
“Diabetic” biscuits full of maltodextrin
Sugary Cereal and bread sold as ‘healthy’
This isn’t nutrition. It’s confusion.
Low-carb challenges the noise and brings back real nourishment
Advocating for progress in mental health isn’t anti-psychiatry.
It’s not a fight against clinicians.
It’s not a fight against science.
Instead…
It’s a fight *for* science.
*For* innovation.
And most importantly, *for* better outcomes and better lives for patients.
People don’t always need answers.
Sometimes, they just need to feel seen, heard, and understood.
Your presence can sometimes be more healing than your advice.
You were born sinless, you are not suffering from any generational curse from your ancestors. Your ancestors were not demons, they were great Africans.