Whatโs next? Language unlocks potential capabilities beyond accuracy:
๐ Open-vocabulary prediction: extend to new diagnoses with minimal tuning.
๐ Natural language slide search: e.g. "slides showing interface dermatitis with basal vacuolization"
๐ฉบ Further enrichment with case-specific clinical context
Full blog post: https://t.co/0WShasbRsj
#DigitalPathology #ComputationalPathology #HealthcareAI #PLUTO4 #MultimodalAI
๐ฌ Moving Beyond Pixels: Advancing Multimodal Pathology
Pathology AI has long relied on images alone. What happens when you add language?
We combined PLUTO-4 vision embeddings with rich histological descriptions to build a joint vision-language space for disease classification, and the results are promising.
๐งต #AI #MachineLearning #Pathology #MultimodalAI #FoundationModels
Looking deeper into the the shared embedding space, it has clinically meaningful structure:
๐ข Subtypes cluster together
๐ค Inflammatory dermatoses separate cleanly from melanocytic lesions
Slide embeddings mirror text embedding organization, showing alignment across modalities, not just within them.
These results highlight how our PLUTO-4 foundation models enhance PathAIโs AI-pathology products across digital diagnostics and translational research.
Weโre excited for the new capabilities PLUTO-4 will unlock for our partners and the community!
๐ Learn more in our technical report:
๐ https://t.co/530ByAxNly
#AI #Pathology #HealthcareAI #FoundationModels #PLUTO4
๐ Excited to share PLUTO-4, our new state-of-the-art foundation models for pathology! ๐ฌ
Weโre seeing SoTA performance across multiple public benchmarks (EVA and HEST) โ surpassing other leading pathology foundation models. (1/6)
#AI#MachineLearning#Pathology #FoundationModels #HealthcareAI
Beyond public benchmarks, PLUTO-4 shows real-world impact โ
๐ฉบ ~10 % improvement across multiple PathAI products, with strong gains in dermatopathology specimen classification.
These advances bring us closer to robust, generalizable FMs for pathology applications.
#Dermatology #HealthcareAI
The opportunity to standardize the way we construct data sets is important...If we try to build a data set for every use case, we can set ourselves up to fail. We don't want to build a large reference data set that doesn't get used - @balasubramaniac from @Path_AI#FriendDx
๐ Interpretable concepts found using SAE
- SAE trained on PLUTO embeddings disentangled polysemantic features. Single dimensions captured distinct concepts:
โ Cell types (e.g., cancer cells, red blood cells)
โ Geometric features (e.g. edge of tissue)
โ Artifacts (surgical ink)
PathAI #MachineLearning engineers have
recently published new #AI findings for mechanistic interpretability of PLUTO, a pathology #foundationmodel. Using sparse autoencoders (SAEs), we uncovered biologically meaningful and interpretable features. ๐งต
https://t.co/4T8GWU7y9p
Monosemantic representations
- Single SAE dimensions correlate with counts of single cell types. For example, SAE-1736 represents plasma cell abundance exclusively
- The findings generalized to:
โ Out-of-domain datasets (CPTAC)
โ Different stains (H&E, IHC)
โ Various scanners
๐ฌ Impact
This study shows the promise & potantial of SAEs in explaining foundation model behavior for medical imaging. Interpretable features unlock:
- Potential for clinical AI ๐ฅ
- New biological insights ๐งช
๐ Read the full work: https://t.co/2R6c7ck4wH
#AI#Pathology