We found the performance of GPT-4 in complex clinical cases to be quite impressive. It will be interesting to follow future developments in AI for diagnostics. We are very excited to have our results published in @NEJM_AI. Thanks to co-authors @JesperRyg and Sören.
@sama@gdb
Early evidence that state-of-the-art generative AI performs adequately on c complex cases [Ed. looking forward to prospective trials] “Use of GPT-4 to Diagnose Complex Clinical Cases” in early release articles of inaugural issue of @NEJM_AI https://t.co/6Uhha29Y4l
Reporting guidelines for use of generative #AI like #ChatGPT in medical research - such as manuscript writing. Happy to be part of the group.
https://t.co/dzbbGHz74O
Augmenting LLM with treatment guidelines for clinical decision support in bp depression - with @RecoveryDoctor + Joe Goldberg + Chris Schneck: https://t.co/QGxH7B4EXM
AI is moving so fast it is insane. Look up some 1 year old AI generated videos for comparison. Will be exciting to see if this works as advertised. If it does, it is truly incredible technology.
Google just made an incredible AI video breakthrough with its latest diffusion model, Lumiere.
2024 is going to be a massive year for AI video, mark my words.
Here's what separates Lumiere from other AI video models:
@sniezenMD@CaseyMcQuadeMD@NEJM If you mean in this particular test, both versions were run with the same cases, and both models ran those cases 5 times.
@emollick I think part of it is due to how fast it is moving. Even when spending a lot of time with this it is hard to keep up just for my own profession not to mention other professions. Another thing is probably bias - "ain't no way no dumb robot can ever beat me" kind of thinking.
Reading this study as a doctor, I feel like I just received adenosine.
Jests aside, skimming this paper it seems like an amazing study. The advent of AI in healthcare cannot be underestimated and it has the potential to seriously alleviate the lack of resources in healthcare and provide equal access to diagnostics for everyone.
@JesperRyg @zakkohane
Superiority of large language model #AI randomized vs 20 primary care physicians on 149 case scenarios for
diagnostic accuracy, conversation, communication, clinical exam skills, empathy, & management plan
https://t.co/RdmBrrXwzb @taotu831@alan_karthi@vivnat@GoogleDeepMind
I touched on the idea of sleeper agent LLMs at the end of my recent video, as a likely major security challenge for LLMs (perhaps more devious than prompt injection).
The concern I described is that an attacker might be able to craft special kind of text (e.g. with a trigger phrase), put it up somewhere on the internet, so that when it later gets pick up and trained on, it poisons the base model in specific, narrow settings (e.g. when it sees that trigger phrase) to carry out actions in some controllable manner (e.g. jailbreak, or data exfiltration). Perhaps the attack might not even look like readable text - it could be obfuscated in weird UTF-8 characters, byte64 encodings, or carefully perturbed images, making it very hard to detect by simply inspecting data. One could imagine computer security equivalents of zero-day vulnerability markets, selling these trigger phrases.
To my knowledge the above attack hasn't been convincingly demonstrated yet. This paper studies a similar (slightly weaker?) setting, showing that given some (potentially poisoned) model, you can't "make it safe" just by applying the current/standard safety finetuning. The model doesn't learn to become safe across the board and can continue to misbehave in narrow ways that potentially only the attacker knows how to exploit. Here, the attack hides in the model weights instead of hiding in some data, so the more direct attack here looks like someone releasing a (secretly poisoned) open weights model, which others pick up, finetune and deploy, only to become secretly vulnerable.
Well-worth studying directions in LLM security and expecting a lot more to follow.
@OpenAI fix this error in playground please; happens when using it seemingly no matter what
thanks ♥
Unrecognized request argument supplied arkose_token
Volume 1, Number 1 is here! Read the inaugural issue of NEJM AI, a new journal on medical artificial intelligence and machine learning from NEJM Group: https://t.co/vte8E7n3K1 📚
🔖 Bookmark for later. Share with colleagues.
Congratulations to the team at @NEJM_AI on the launch of the new journal! 🥳
I am honored to see our work on #GPT4 featured in the journal. It has been a great experience working with them. Hope it can be repeated.
The journal is available at https://t.co/7lWDZadoU3
It will be interesting to see what the future holds for #AI in healthcare.
Our article is now permanently available (with supplementary) at https://t.co/6aUEOAoGqD
@JesperRyg