🎓 Finally had my Bachelor's conferring ceremony in Biomedical Engineering from UCD, 4 years after graduating with a 4.07 GPA, and finishing top of my class🏅! Grateful for the support of family, mentors, and friends. Onward to clinical rotations! 💪
#UCD#Engineering#Medicine
Thrilled to share that our paper on AMIE—our conversational medical AI—is now published in @Nature. We wanted to tackle one of the most challenging areas in clinical AI: management reasoning over multiple patient visits.
https://t.co/03A1pipFI8
Officially a medical graduate from @ucddublin ! 🎓
Thankful for my family, friends, and mentors who helped me get here.
Excited to start my internship at @Mater this July and especially looking forward to my Cardiology rotation. 🫀🩺
Can AI read an ECG like a cardiologist - from just an image?
We built ECG-GPT, a vision-text transformer that generates complete diagnostic reports directly from photos of 12-lead ECGs
Now out in @ESC_Journals#EHJDigitalHealth
Kudos to @aakhunte & @Veer_Sangha_ for leading this @cards_lab
🧵
Animations were sourced from @CompleteAnatomy, essential for my anatomy modules during preclinical years at @UCDMedicine. I edited them in #FCP and added custom infographics from the "Youtube Essential Library" bundle I bought from @envato. These tools made it all come together!
While reviewing cardiac anatomy for my clinical rotations at @svuh, I came across a coronary circulation presentation I created in my first year.
I’m proud of the work that went into this project! I hope it proves helpful to others!
#HeartAnatomy#Cardiology#MedEd
Starting clinical rotations at @svuh this week!
Excited to apply what we've learned and grow as future doctors. Big thanks to our preclinical lecturers at @UCDMedicine for preparing us for this journey. 🏥🩺
#ClinicalRotations#MedicalSchool#UCD
🎉 Wishing Dr. @cyrilzakka all the best in his new role at Hugging Face. His mentorship during his fellowship at Stanford had a lasting impact on me. Looking forward to continuing our work remotely during my clinical rotations! 🩺
This summer, I had the privilege of returning to Professor William @HiesingerLab at @StanfordMed, working on AI models for echocardiograms 🫀👨💻. It was an amazing opportunity to continue my research from last year!
#AIInHealthcare#StanfordMedicine
🌟 A big thank you to Dr. @mrudangm14 for his invaluable insights while working on my first-author paper. His expertise and encouragement were incredibly helpful!
Exciting week! We're also releasing an early preview of MediSyn, a family of text-diffusion models exploring the potential for generative models to synthesize medical data across many modalities (2D and 3D) and specialties (dermatology, radiology, pathology, GI, ophthalmology, surgery and more)
We're still exploring different techniques to improve text alignment and generation quality but this is a great first step!
Our research on ‘Almanac: Retrieval Augmented LLMs for Clinical Medicine’ is finally out on @NEJM_AI! We demonstrate that LLMs augmented with tools (browser, calculators) outperform ChatGPT-4 & others on a novel dataset of 300+ questions assessed by board-certified physicians!
I’m excited to finally showcase our work on developing a generalizable deep learning system for cardiac MRI. This is the culmination of over 3 years of research spanning my time as a postdoc in @HiesingerLab and now as a cardiac surgery resident at @pennsurgery, and I couldn’t be happier with the finished product. Weights to be released for academic use (coming soon!). All built with @pytorch and @pytorch_lightning.
Lots of really cool stuff to unpack, but here are few main results: Contrastive learning with cine-sequence CMR studies from multiple different views (short axis, 2-chamber, 3-chamber, and 4-chamber views) with long-form CMR free text reports is really hard. A good sign for us was how the video transformer embeddings evolved with each epoch. We train for about 600 epochs taking about 2 weeks.
In a zero shot fashion, the model can then do some really interesting things: separate out different diseases (ACDC dataset), and different genders, age groups, and ejection fractions (UK BioBank) with no explicit supervised instruction. No such behaviour with kinetics-600 initialized weights (see preprint)!
Finetuning over our system yields results superior to baseline approaches on downstream tasks of interest, with 10x and sometimes 100x less data. Here’s a quick figure showing how we achieve superior results for UK BioBank LVEF estimation with just 1% of the available data. Better yet, that same model when tested on a Kaggle dataset does exceptionally well (dashed line = kinetics-600 performance on left; lower is better!).
Focusing on some disease diagnosis tasks, we label a separate dataset of some 6000 individual patients. This is real-world representative unenriched dataset. Our methods show massive improvements in AUC compared to baseline methods. Validation set shown here, internal test set (Stanford / Medstar / UCSF), and external test set (UPenn) figures are in the paper.
There’s been a lot of buzz surrounding multimodal LLMs in medicine. At the @HiesingerLab we’ve been building Almanac Chat for over 6 months alongside our collaborators and we’re excited to showcase the first preview!
• Almanac Chat is the first truly multimodal medical LLM trained across specialties and modalities (i.e. images, text, video and audio) to serve as a generalist clinical copilot.
• The entire stack (e.g. server, client, ML pipeline) is developed and hosted locally to guarantee privacy and flexibility at inference time.
• We're looking forward to evaluating our model alongside domain experts at @StanfordMed in Q2 2024.
• We have a a lot more exciting things on the way - see if you can spot a few teasers in the video!
👥: We’re looking to scale our team, model, and hardware. If you're interested in a partnership or working alongside myself and our recent hire @joseph_cho1 please get in touch!
Ever wanted to select all items in your @Zotero library that you've cited in a certain Word document?
Reference Extractor now has a "Select cited items in Zotero" button to do just that. https://t.co/zNqFvDMhVH