Excited to be at #AMIA2024 in SF this week, giving two talks (both today!):
âĄď¸ Evaluating Large Language Models for Drafting ED Discharge Summaries
âĄď¸ Utilizing GPT-4 to determine reasons for missed follow-up colonoscopy following abnormal non-invasive colorectal cancer screening
#AI is helpful in an ER setting but shouldn't be blindly trusted, UCSFâs Dr. @cykwilliams tells @kron4news. It can help answer exam questions & draft notesâ"but itâs not currently designed for situations that call for multiple considerations," he explains. https://t.co/plGZu3nQEr
Thrilled to announce our latest paper 'Evaluating the use of large language models to provide clinical recommendations in the Emergency Department' published today in Nature Communications!
https://t.co/1MmGIoD1t6
Grateful for the opportunity provided by the NAIRR Pilot program! I hope it becomes a permanent program to democratize access to computing resources. Extending thanks to @bmeow19 for helping all through the ideation and writing, and Oksana for helping shape it even further!
WOW! @MadhumitaSushil wins another National Artificial Intelligence Research Resource (NAIRR) pilot award, now bringing @UCSF total to 5 and the @UofCalifornia total to 16! BRAVO!
https://t.co/tn8k3KLgIT
And it's out! Congratulations @madhumitasushil and team on our new publication in Journal @AMIAinformatics on comparing multiple LLMs on zero-shot inference and task-specific supervised classification of @UCSF breast cancer pathology reports, including UCSF-BERT!
"On tasks where large annotated datasets cannot be easily collected, LLMs can reduce the burden of data labeling"
https://t.co/JtcMg7cIBB
If youâre currently having trouble accessing ChatGPT, Claude, or other platforms⌠you know where to go.
https://t.co/Uq2XrTpjTO systems are alive and well.
I recently had the privilege of presenting some words at the 2024 Graduation Commencement of the @McWilliamsSBMI (thanks for the invite, Dean @JiajieZhang100)!
Here is the speech I delivered, if anyone is interested:
Dear parents, families, faculty members, staff, and of course the 54 distinguished graduates celebrating with us here, out of the 90 total.
You are not just getting a degree today, you have somehow managed to be the very best experts in getting computers to understand humans, and humans to understand computers, in the especially difficult area of biomedicine!  Youâve mastered the art of algorithms and analytics and artificial intelligence. Youâre the magicians who turn data into diagnoses, treatment plans, and pathway analyses, and sometimes, even more data!
But to be serious, we are privileged to be in this field. We are lucky to be alive when computers are so incredibly powerful and data from protons to patients to planets are so available, and even open access!  It may not seem so, but we are lucky to be able to write grant proposals and get a few of them funded. It is fun and sometimes funny to complain on social media when NIH doesnât fund the very best grant proposal weâve ever written. But we are privileged. In most cases, we still get to keep our jobs as much needed informaticians, in academia or industry, when budgets or proposals donât get funded. Letâs remember how lucky we are all to be here.
Today, I really want to communicate my thoughts about duties. I think of four duties we have as informaticians.
The first duty is a Duty to help others: especially those we call patients or participants.  The importance of health data has never been stronger for the entire worldâs population. It could not be more clear how much of the rest of the worldâs education, commerce, transport, exchange, communication, prosperity, and livelihoods all depend on health. And as weâve seen, the opening and closing of the world for business depends on a daily streams of numbers.
Biomedicine, and all the industries that it touches, is already a data-driven business, but letâs not lose the humanity behind it.  And itâs important for all of our organizations to take care of the data, respect it, protect it, but also to use it, to help our patients and families. Respect the data and especially respect those that luckily let us have their data, samples, and measurements.
Hereâs another way to think about our duty to help others. How many of you were funded on a federal grant? Like a training grant, or your lab principal investigatorâs NIH grant?
Why do you think the federal government gives out grants to us? I donât think they just give us money for us to get a great job as a scientist. I think they give us money for us to create jobs too!  When the government says we want to create jobs, how do you think they do it? We are part of that solution!
Look, many of you are looking to get a job either in industry or academia for your next steps. But in your future steps, I am asking you to think about how to create even more jobs and perhaps start more companies! I believe that is a big reason why the government gives us money to pursue our ideas is to empower us to help our communities by creating careers.
The second duty is a Duty to the truth: we are strangely in a reproducibility crisis, a false data crisis, and a plagiarism crisis, all at the same time! And we have a lot of competing interests in healthcare. Some of you might end up working at a health care payer. Some of you might end up at a health care provider. IF we are not careful, your AI is going to fight their AI!
But go after the big problems! A small solution for a really tough problem is still going to help more than yet another way to analyze the same type of data again and again. And donât just build a web-site. If youâve built a computational tool, you are already the best at using it! Use that tool on data, and if you canât find data, use open public data, and teach the world what youâve found, not just the mechanics of your tools.
I do very much believe that the vast majority of scientists and engineers do well and mean well. Donât fall into traps like believing your career depends on H-index or citation counts. These are made up games. Did Marie Curie or Albert Einstein even care what their citation count was? iâm guessing not.
The third duty is a Duty to yourself in the end, you are responsible for your health, physical and mental. They say we only get one body, so watch out for it. itâs easy to spend hours in our profession sitting in front of a computer. Please remember to seek out the outside and walk, yes, even though it is impressively and oppressively humid here in Houston!
Finally, the fourth duty is a Duty to those who got you here: Theyâve invested time, money, effort, and in most cases love, and we owe them our thanks. They are counting on you to change the world.  I would love for the graduates to stand and thank those that are here and beyond, for getting us here, all together for this moment.
Weâre in a time when we, in general, are incredibly privileged to be here with our education and our potential, but we are admittedly living in complicated times, where even universities around us are trying to reestablish their roles in education, public discourse, or even as the town square.  I do encourage you to try to seek out the goodness in the news, in your science and engineering, in your neighbors, and in your colleagues. We are all in this together, we get one big chance at it, letâs all try to do some good for this world.
Thank you.
@LiamGMcCoy@UCSF@UCSF_BCHSI Thanks đ and great question! Thereâs definitely a lot of patient preference data in the notes but depends a lot on what clinicians choose to document
Ambient scribing might help pick up more of these patient choices though!
Itâs been a fun journey working in the clinical LLM space đ Come check out my PhD thesis seminar @ucsf Byers Auditorium Thursday 2pm or message me for a zoom link!
@UCSF_BCHSI
PhD exit talk by Brenda Miao @bmeow19 happening on Thursday 2pm @UCSF Byers Auditorium, with reception to follow! Join us in celebrating Brenda's PhD work!
Uncovering Strategies For Personalized Treatment Selection Using Large Language Models
Healthcare data has never been so accessible to patients and physicians, from smartphones and other remote monitoring devices to improved Electronic Medical Record (#EMR) data access. Despite this increasing ubiquity, insights from these data are often only captured in medical notes and other complex, spare, and unstructured real-world data. Here, we develop methods to use large language models (#LLM) to identify points of actionable insights from real-world data for both digital and pharmaceutical therapeutics. These new methods allow us to take an unprecedented look at the conversations, decisions, and medical expertise captured in millions of clinical notes and other real-world text data. #AI
Such a pleasure to be live on the @abc7newsbayarea evening news discussing how AI might one day be used in #ER triage! Check out the replay here if you missed it (from 1.55): https://t.co/R389b2SPrR @UCSFHospitals@UCSF_DOCIT@atulbutte