🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks.
The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people.
ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher.
My Take
The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing.
I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all.
Hedgie🤗
https://t.co/8Kg8FOrgHW
@JamieBonkiewicz@gtconway3d The Sharpie-enhanced map showing the big, beautiful new alternative route for oil tankers. No one’s ever seen anything like it.
@sadebugo@BretWeinstein Has anything changed in your lifestyle that would decrease your exposure? For example, when teachers retire, they often have fewer colds/flu, because they’re not around kids all day.
@RFK1966@WalshFreedom Exactly, came here to say this. Gerrymandering exacerbated extremism. Candidates had to out-crazy each other in every primary, I watched it happen in my state.
@RC_Cola_Fan@Penny_J_Thomas Iranians are not Arabs, Persia was never conquered by the Arabs. There are Arabs (and Christians, and Jews, and Kurds, etc.) living in Iran, but the main ethnic group in Iran is Persian.
@adonipriapus@OfAthenry@JoyceWhiteVance I didn’t say it was ONLY your opinion, I said it was an opinion, not a fact. And you are still pretending that because I said enforcing federal law is my favorite thing DOE does, that that is the only thing of value that they do (in my opinion).
@LevelOneRaccoon@DoritoReiss@OfAthenry@JoyceWhiteVance I didn’t say they don’t do anything, and I didn’t say enforcing federal law is the only thing they do. I said it was my favorite thing they do. I’m concerned that the replies I’m receiving indicate to me that building straw men is the only response you all can come up with.