ARISE, a Stanford- and Harvard-backed research network, just released a very comprehensive report that looks across the evidence to understand what's happening in industry as AI moves into clinical practice.
Here are the 6 key themes:
1. Healthcare has moved past pilots. More than 80% of healthcare leaders have deployed at least one AI use case, making healthcare one of the fastest-moving industries for adoption
2. Great benchmarks don't guarantee great products. Models continue to post impressive scores on medical exams, but performance drops once they enter real clinical workflows with incomplete data, interruptions, and messy edge cases.
3. Safety is improving, but there's still work to do. On the NOHARM benchmark, even leading models produced errors in about 1 of 14 clinical consultations. Most came from missing important information rather than making something up.
4. AI works best when it handles the busywork. Documentation, inbox management, coding, and other admin tasks continue to show clearest ROI. More autonomous clinical workflows are improving, but still have a higher bar to clear.
5. Patients are moving faster than the system. AI is becoming a routine source of health information for consumers, with adoption up ~2x over the past year. Healthcare organizations are still figuring out how to respond.
6. The biggest challenge now is trust. Models will keep getting better. The harder question is which companies can prove their tools are safe, reliable, and actually improve care once they're deployed.
I have come to hate the AI cadence of juxtapositional/antithetical style. “This isn’t a this—it’s a that!”
Laura, you don’t have an AI problem, you have a problem with AI! And your problem isn’t with the solutions themselves—it’s solving for the problems that the solutions create!
It also depends on environment. Recording in a hospital could inadvertently violate another patients rights—many rooms are shared by more than one patient. In general when patients want to record or have me on the phone with a family member it’s because they want to remember what I said so everyone is on the same page. It can be really helpful for patient care. My wife has called me from private physician offices and told the provider she wanted her husband, a physician, to listen in and she’s had several doctors refuse—one even told her to hang up the phone when I answered the call. That to me sounds more like authoritarian care than collaborative care and no patient will remember everything that is said in an encounter. On the other side, as a remote consultant I’ve had ER doctors facetime me with patients consent so i could explain their findings directly, especially in complex cases they werent familiar with.
That’s a bit inaccurate. Yes, we can image through thin bone—and it’s done all the time. We read transcranial Doppler regularly. But I agree that it’s disingenuous to compare that simulated wave propagation video to real world diagnostic US, even TCD. It’s just not like that at all. We need to be fair minded here and understand that the limits of technology are current limits and not immutable laws. I won’t be surprised if we can get better transcranial resolution with better computation and algorithms. But I don’t think we’re there today based on anything I’ve seen. Happy to be surprised. Outside of that, whole body US is intriguing and if you’re 360 image acquisition you map out the bones pretty nicely. Their raw compute power is insane and I’m not surprised that they can generate a good image. But I doubt it’s current diagnostic yield.
I think what we have failed to communicate or what has failed to be understood is that our management algorithms are designed with evidences based medicine best practices. They evolved with the technologies. And without an epidemiology degree or medical degree most don’t understand that even a new screening technology doesn’t bridge the gap to diagnostic evaluation. And broad screening, as you said, leads to higher mortality and costs due to the adverse effects of workup and the number of additional high specificity exams required to get to the answer. Maybe in a near future state we’ll create tests that are both 100% sensitive and specific for each disease, but that isn’t the reality today. The reality is more harm is caused by whole body screening programs than benefit.
The thing that we sometimes miss is that the harm
And benefit occur to different people. So the person who had a cancer found with screening is thanking god for the test. The person who dies as a result/unavoidable complication of those work ups or at a minimum pays the thousands and experiences unnecessary radiation etc, will typically be a different person. It’s not that there is no benefit, it’s that there is no net benefit and currently more harm.
Always great thoughts Laura! I totally agree. I am excited to see new diagnostic solutions come to the market and I hope they can really dial this in for diagnostics. I’m all for innovation but I agree this is a first step into an inevitable direction without much evidence or research available beyond the hype. But sometimes hype can be fun.
That’s really the question. The technology itself is potentially fantastic and inevitable. My question is how do we use this for diagnostics? Outpatient? Inpatient? Tons of potential, but it needs to be investigated and compared to current standards. I’m optimistic at the trajectory if not the implementation.
I described this potential tech to my friends in residency 15 years ago and I got some texts today saying “someone built it”. I’m glad to see we finally have the compute to accomplish it. I’m wondering if you’ll have contrast enhanced procedures, if you’re accommodating for disability, pregnancy, etc (like a seated water bath). Will you use this for diagnostic radiology or if you’re only intending it as a screening tool to compete with wbmri? I think the diagnostic capabilities are really the most exciting.