Jian W Dong, MD/MS

Radiologist & tech optimist, mostly interdisciplinary takes ← MS DS/AI, UT Austin ← Radiology , Harvard ← MD, NYU ← MechE + Neurosci, Columbia ← the hood

[Jian W Dong, MD/MS @JianWDong]

Try it out for yourself: https://t.co/zrRsqf5i0Q

No nvidia GPU needed; this can run on a macbook pro. Takes 30-60 seconds for an uncompressed inference on a macbook pro (ymmv, mine is a 3-year old macbook pro with m2 max).

[Jian W Dong, MD/MS @JianWDong]

My review...

Overall it's the most impressive general CT model I've seen so far.

BUT! I would say it's still not very good. Maybe at the level of a first year radiology resident.

I tested it on a few easy cases and in each case the model has hallucinated findings (e.g. lung nodule that doesn't exist, radiation fibrosis that doesn't exist, foley catheter in the urinary bladder on a CT chest that did not include the urinary bladder) or missed findings. See example image below.

[Maziyar PANAHI @MaziyarPanahi]

70,111 CT scans trained this model. one NVIDIA GPU turned a full chest CT into a radiology report in under 30 seconds 🔥

top finding: a spiculated nodule, right upper lobe. the expert's tumor marking it never saw? right lung.

open weights from @nvidia on @huggingface 🤗

[Jian W Dong, MD/MS @JianWDong]

@GRomyll It's def interesting. I actually uploaded CTs from publicly available datasets. There are a ton available.

I'd be curious how it does on more rare and more well-defined diseases (e.g. cystic lung diseases, UIP, etc).

[Jian W Dong, MD/MS @JianWDong]

@MaziyarPanahi I think having a dicom viewer embedded within the browser is helpful. You can evaluate the AI findings by looking at the CT images directly on the same page.

[Jian W Dong, MD/MS @JianWDong]

Thanks. Totally get it's not your model. No need to share with the authors because this doesn't constitute an "error" on their paper. I am just opining on my evaluation of the model, and quoted tweeted yours because your tweet was piqued my curiousity.

I didn't use a quantized version, but I get why you would think that. This is the full version, just running locally on my macbook pro with a radiologist-friendly interface (soft tissue/lung windowing etc). I did this because this is the way for a lot of non-technical people to test it out. Lots of radiologists would be interested in it, but don't have nvidia GPUs. A souped-up macbook pro with enough apple silicon should be able to run the full version...I've been doing NN training and inference on my macbook pro for years.

My (Claude's) code is here: https://t.co/bOSGLaya2e

[Jian W Dong, MD/MS @JianWDong]

One of the most notable examples of patients not caring about how a cure works as long as it works is deep brain stimulation surgery. Two long electrodes implanted deep into the brain to treat Parkinson disease. Even after decades of research, no one really knows the underlying mechanism. It was discovered serendipitously, too.

[Andrew Curran @AndrewCurran_]

This is a slide from Terence Tao's 'Math 2.0' public lecture yesterday evening at Caltech. I am highly confident that, if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is completely irrelevant to them.

[Jian W Dong, MD/MS @JianWDong]

@IncidentalMD @francisdeng This is why I don't show my toddlers AI-generated photos or videos. Need to ground them in reality first.

[Jian W Dong, MD/MS @JianWDong]

I think the opposite has happened: radiologists are reading more in spite of AI, not because of it. He previously said that AI in radiology has saved radiologists time, allowing us to read more. That’s not my experience and intuitively it cannot be true based on the current ai-in-radiology workflow: A radiologist currently still reviews the entire scan the way we have always done pre-AI, and then now on top of that needs to review the AI findings (many which are false positives, and then decide whether or not to incorporate it into the report). On a time-basis any AI finding is additive, not substitutive.

These newer foundational models could help with report generation though.

[Jian W Dong, MD/MS @JianWDong]

@DevanSinha The Mark Cuban Cost Plus drugstore is actually run by a radiologist

[Jian W Dong, MD/MS @JianWDong]

Uh. The more appropriate way to frame this is text-based AI is much easier to learn than physical or geospatial AI, because of the properties of the underlying attention mechanism. This is why you have AIs that can solve Olympiad math problems but not reliably fold your laundry. By the way self-driving is nowhere near solved. @karpathy talked about this in one of his appearances in a podcast. Listen to him. He's very grounded in reality and a technical expert.

[Jian W Dong, MD/MS @JianWDong]

Name one AI, commercial or research, that's capable of fully interpreting a study end-to-end. By that I mean it will interpret a study whether it is a CT of the abdomen and pelvis or a chest radiograph, comparing it to any relevant priors, and put forth an accurate clinical radiology report.

[Ry Lanham @ry4lanham]

I think we both know this is a game about legal liability. Doctors refuse to hand over autonomy if they remain liable. The AIs can easily do the work. Easily. Indeed, over 50 pct of US hospitals already use AI in their radiology workflows. Stepping to 100 pct is trivial technologically.

[Jian W Dong, MD/MS @JianWDong]

@txmedai @grok Evaluate each link one by one. Do any of these approximate a general AI radiologist described in the original tweet?

[Jian W Dong, MD/MS @JianWDong]

What do you mean by AI reading scans? Is there software or research showing general complete autonomous AI reading of a certain type of imaging study?

Current AI image interpretation in radiology is extremely narrow (e.g. hemorrhage or no, LVO or no, pulmonary nodule or no, PE or no, fracture or no). They are not comprehensive interpretations of the scan - for even the simplest modality such as an x-ray of the chest there are 100s or 1000s of pathologies, across various scan conditions and image qualities, temporal comparisons, etc. You would need to run 100s or 1000s of algo for a CXR. Perhaps transformer based foundation models may change that....but...I'll save that critique for another day.

Not trying to sound pessimist. I'm quite bullish re: AI in medicine. Just gotta be grounded in reality.

[Jian W Dong, MD/MS @JianWDong]

"Excluding radiology, a domain in which AI alone does not reliably exceed physicians, only 9 studies published since January 1, 2024, concluded that physicians alone are equivalent or superior to AI alone at 5 fundamental cognitive medical tasks".

Lol.

A decade ago, the authors were quite confident that the radiologist would be the first to be replaced by AI.

[Neal Khosla @nealkhosla]

1/ We started @CuraiHQ with the mission of proving AI could outperform humans at the core cognitive work of medicine.

Today in @JAMA_Current, @ZekeEmanuel, @AbeBakerButler, @vkhosla, and I are publishing a comprehensive argument this future is here.

https://t.co/bpXMdXljsw

[Jian W Dong, MD/MS @JianWDong]

@ZekeEmanuel @grok what did he say about radiologists? Did his predictions come true?

[Jian W Dong, MD/MS @JianWDong]

@FCademartiri Accurate. The attention mechanism is much better at text than at vision.

[Jian W Dong, MD/MS @JianWDong]

@ylecun @pmddomingos https://t.co/HvNkFLEE2A 😬

[Jian W Dong, MD/MS @JianWDong]

@RezaC1 @grok what are the consequences of long term cumulative exposure to high p.m 2.5?

[Jian W Dong, MD/MS @JianWDong]

@tobi Horos and IMV viewer are both free on the Mac. Vibe coding is cool too :)

[Jian W Dong, MD/MS @JianWDong]

@tobi That’s not L3-L5.