Joseph Paul Cohen

CompSci + ML + Healthcare. @AWSCloud, @StanfordAIMI, @Mila_Quebec. Director of and

[Joseph Paul Cohen @josephpaulcohen]

Want to understand why a computed tomography classifier made a prediction? The code just went online for generating counterfactual explanations for 2D/3D CT classifiers! Here is an explanation for Plural Effusion: #radiology #radiologyai @StanfordAIMI

https://t.co/1Kh3tCQyMP

[Joseph Paul Cohen @josephpaulcohen]

My journey learning about token level diffusion LMs at #neurips25

[Joseph Paul Cohen @josephpaulcohen]

At @NeurIPSConf + @SymposiumML4H next week! What's the word on XAI and deploying AI in healthcare?

[Shawn Tan @tanshawn]

We're looking for 2 interns for Summer 2026 at the MIT-IBM Watson AI Lab Foundation Models Team.

Work on RL environments, enterprise benchmarks, model architecture, efficient training and finetuning, and more!

Apply here: https://t.co/hS2meJm9j4

[Josh Myers-Dean @Joshmyersdean1]

VizWiz is back at #CVPR2025! Come join us for fun challenges and an exciting speaker line up! Details can be found at: https://t.co/yLi699kKPV @CVPR

[Joseph Paul Cohen @josephpaulcohen]

"Any measurement that you make without the knowledge of its uncertainty is completely meaningless" -Walter Lewin

Please consider this when you put a table full of numbers in your paper!

[Joseph Paul Cohen @josephpaulcohen]

Poster #280!

[Joseph Paul Cohen @josephpaulcohen]

Attending #CVPR2024? Check out our method for identifying spurious correlations in neural networks using counterfactuals at the #XAI4CV workshop on Tuesday!

https://t.co/tYH9TtxQY9
https://t.co/vMI97LCVlv

@CVPR @CVPRConf @XAI_Research

[Joseph Paul Cohen @josephpaulcohen]

This is pretty cool! https://t.co/oHevS9hX0S
Just in the onboarding it is finding interesting papers I wasn't aware of!

[Joseph Paul Cohen @josephpaulcohen]

@pranavrajpurkar Here we look at the effect of counterfactuals in avoiding false positives of an AI assistant. 2 radiologists x 240 images = 480 data points.
https://t.co/BNk9Jzni5n

[Pedro Domingos @pmddomingos]

Proof that consciousness doesn't exist:
- An atom isn't conscious.
- Adding an atom to something that's not conscious doesn't make it conscious.
- Therefore nothing is conscious.

[Joseph Paul Cohen @josephpaulcohen]

@Joey_Wittmann @academictorrent @comma_ai I use that code to manage the donated seedboxes we have. So I'd say it is useable. Although it could be more polished (PRs welcome!). You can maintain a collection on the site and add that collection into the smartnode and it will download them.

[Joseph Paul Cohen @josephpaulcohen]

@cataluna84 @academictorrent All seeding nodes are shown on the map. You can get your account photo shown on the side as a "Hosted by" attribution by downloading a torrent file linked to you if you are logged in when you download it. https://t.co/jIhIaSm3Cz

[Academic Torrents @academictorrent]

Grok-1, 3 days, 300GB, 5638 downloads, 1.6PB downloaded, 8.12MB/s average download speed. Here is a map of the 2000 hosting locations!
#academictorrents
https://t.co/XkcK5WRyn9

[Sunnie S. Y. Kim ☀️ @sunniesuhyoung]

The XAI4CV workshop will be back at #CVPR2024 led by the amazing @indu_panigrahi!

https://t.co/6QE84yj0l5

This year the organizing team spans 8 institutions @sukrutrao @KolekDe @LenkaTetkova @_Kate_Morrison_ @deeptigp + Jawad, Pushkar, Vikram ❤️

@CVPR @CVPRConf @XAI_Research

[Machine Learning for Healthcare @mlforhc]

MLHC 2024 Call for Papers is now open!
https://t.co/uS30VHqfEf

[Machine Learning for Healthcare @mlforhc]

Registration is open for MLHC 2024 in Toronto! Register today to reserve your space!
https://t.co/vJ3I7R12aC

[Joseph Paul Cohen @josephpaulcohen]

Vision-language conversational models for healthcare may fit a use case with physicians better than task-specific tools. How to validate them remains a challenge but the conversational interface may be the key to get adoption by physicians.

[AK @_akhaliq]

CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation

paper page: https://t.co/kWW7zSyq0p

Chest X-rays (CXRs) are the most frequently performed imaging test in clinical practice. Recent advances in the development of vision-language foundation models (FMs) give rise to the possibility of performing automated CXR interpretation, which can assist physicians with clinical decision-making and improve patient outcomes. However, developing FMs that can accurately interpret CXRs is challenging due to the (1) limited availability of large-scale vision-language datasets in the medical image domain, (2) lack of vision and language encoders that can capture the complexities of medical data, and (3) absence of evaluation frameworks for benchmarking the abilities of FMs on CXR interpretation. In this work, we address these challenges by first introducing CheXinstruct - a large-scale instruction-tuning dataset curated from 28 publicly-available datasets. We then present CheXagent - an instruction-tuned FM capable of analyzing and summarizing CXRs. To build CheXagent, we design a clinical large language model (LLM) for parsing radiology reports, a vision encoder for representing CXR images, and a network to bridge the vision and language modalities. Finally, we introduce CheXbench - a novel benchmark designed to systematically evaluate FMs across 8 clinically-relevant CXR interpretation tasks. Extensive quantitative evaluations and qualitative reviews with five expert radiologists demonstrate that CheXagent outperforms previously-developed general- and medical-domain FMs on CheXbench tasks. Furthermore, in an effort to improve model transparency, we perform a fairness evaluation across factors of sex, race and age to highlight potential performance disparities.

[Zhihong Chen @zhjohnchan]

⭐️ Excited to share our latest work about AI in healthcare. We present CheXagent, a foundation model for Chest X-ray interpretation.

📄 Paper: https://t.co/GIxXNwK7N9
🌐 Website: https://t.co/rrbR2zHNNy

🧵 1/N

[AK @_akhaliq]

CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation

paper page: https://t.co/kWW7zSyq0p

Chest X-rays (CXRs) are the most frequently performed imaging test in clinical practice. Recent advances in the development of vision-language foundation models (FMs) give rise to the possibility of performing automated CXR interpretation, which can assist physicians with clinical decision-making and improve patient outcomes. However, developing FMs that can accurately interpret CXRs is challenging due to the (1) limited availability of large-scale vision-language datasets in the medical image domain, (2) lack of vision and language encoders that can capture the complexities of medical data, and (3) absence of evaluation frameworks for benchmarking the abilities of FMs on CXR interpretation. In this work, we address these challenges by first introducing CheXinstruct - a large-scale instruction-tuning dataset curated from 28 publicly-available datasets. We then present CheXagent - an instruction-tuned FM capable of analyzing and summarizing CXRs. To build CheXagent, we design a clinical large language model (LLM) for parsing radiology reports, a vision encoder for representing CXR images, and a network to bridge the vision and language modalities. Finally, we introduce CheXbench - a novel benchmark designed to systematically evaluate FMs across 8 clinically-relevant CXR interpretation tasks. Extensive quantitative evaluations and qualitative reviews with five expert radiologists demonstrate that CheXagent outperforms previously-developed general- and medical-domain FMs on CheXbench tasks. Furthermore, in an effort to improve model transparency, we perform a fairness evaluation across factors of sex, race and age to highlight potential performance disparities.