🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks.
The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people.
ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher.
My Take
The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing.
I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all.
Hedgie🤗
https://t.co/8Kg8FOrgHW
Tremendous turnout for Katie Wilson’s inauguration at Seattle City Hall… up the stairs, hanging over the second floor.
Streaming now on @komonews dot com.
"But if H5N1 were to be allowed to run through a flock of five million birds, 'that’s literally five million chances for that virus to replicate or to mutate,' Dr. Hansen said."
#medlibs#veterinarians https://t.co/GcbyrG7Uk1
Here's one I didn't know about. "U.S. National Institutes of Health-related updates to the Federal Register, which are required for the scheduling of study sections and advisory councils, are on hold indefinitely." #medlibs https://t.co/3fQsidMzoG
#medlibs "This is not how it works. Medically relevant terminology and inclusive language follow evidence based reporting standards or are matters of individual journal style and policy. They do not follow political orders." https://t.co/bgkbwUVwP8
"Conclusions: These indexes have potential utility in qualitative search strategies, if only for supplementing other database searches with unique records." #medlibs
Check out this article on database coverage for qualitative information retrieval from my colleagues Jennie, Dave, and Danielle!
#medlibs#canmedlibs#ukmedlibs
https://t.co/uWktSTNuie