What’s next? Language unlocks potential capabilities beyond accuracy:
🔓 Open-vocabulary prediction: extend to new diagnoses with minimal tuning.
🔍 Natural language slide search: e.g. "slides showing interface dermatitis with basal vacuolization"
🩺 Further enrichment with case-specific clinical context
Full blog post: https://t.co/0WShasbRsj
#DigitalPathology #ComputationalPathology #HealthcareAI #PLUTO4 #MultimodalAI
🔬 Moving Beyond Pixels: Advancing Multimodal Pathology
Pathology AI has long relied on images alone. What happens when you add language?
We combined PLUTO-4 vision embeddings with rich histological descriptions to build a joint vision-language space for disease classification, and the results are promising.
🧵 #AI #MachineLearning #Pathology #MultimodalAI #FoundationModels
Looking deeper into the the shared embedding space, it has clinically meaningful structure:
🟢 Subtypes cluster together
🟤 Inflammatory dermatoses separate cleanly from melanocytic lesions
Slide embeddings mirror text embedding organization, showing alignment across modalities, not just within them.
These results highlight how our PLUTO-4 foundation models enhance PathAI’s AI-pathology products across digital diagnostics and translational research.
We’re excited for the new capabilities PLUTO-4 will unlock for our partners and the community!
📄 Learn more in our technical report:
👉 https://t.co/530ByAxNly
#AI #Pathology #HealthcareAI #FoundationModels #PLUTO4
🚀 Excited to share PLUTO-4, our new state-of-the-art foundation models for pathology! 🔬
We’re seeing SoTA performance across multiple public benchmarks (EVA and HEST) — surpassing other leading pathology foundation models. (1/6)
#AI#MachineLearning#Pathology #FoundationModels #HealthcareAI
Beyond public benchmarks, PLUTO-4 shows real-world impact —
🩺 ~10 % improvement across multiple PathAI products, with strong gains in dermatopathology specimen classification.
These advances bring us closer to robust, generalizable FMs for pathology applications.
#Dermatology #HealthcareAI
The opportunity to standardize the way we construct data sets is important...If we try to build a data set for every use case, we can set ourselves up to fail. We don't want to build a large reference data set that doesn't get used - @balasubramaniac from @Path_AI#FriendDx
🔍 Interpretable concepts found using SAE
- SAE trained on PLUTO embeddings disentangled polysemantic features. Single dimensions captured distinct concepts:
✅ Cell types (e.g., cancer cells, red blood cells)
✅Geometric features (e.g. edge of tissue)
✅ Artifacts (surgical ink)
PathAI #MachineLearning engineers have
recently published new #AI findings for mechanistic interpretability of PLUTO, a pathology #foundationmodel. Using sparse autoencoders (SAEs), we uncovered biologically meaningful and interpretable features. 🧵
https://t.co/4T8GWU7y9p
Monosemantic representations
- Single SAE dimensions correlate with counts of single cell types. For example, SAE-1736 represents plasma cell abundance exclusively
- The findings generalized to:
✅ Out-of-domain datasets (CPTAC)
✅ Different stains (H&E, IHC)
✅ Various scanners
🔬 Impact
This study shows the promise & potantial of SAEs in explaining foundation model behavior for medical imaging. Interpretable features unlock:
- Potential for clinical AI 🏥
- New biological insights 🧪
🔗 Read the full work: https://t.co/2R6c7ck4wH
#AI#Pathology