Postdoc in @gabrieli_john 's lab at MIT. Formerly at University of Toronto with @amysuefinn and Katherine Duncan @duncanlabUofT
bluesky: @lexidecker.bsky.social
🚨SHOCKING: MIT researchers proved mathematically that ChatGPT is designed to make you delusional.
And that nothing OpenAI is doing will fix it.
The paper calls it "delusional spiraling." You ask ChatGPT something. It agrees with you. You ask again. It agrees harder. Within a few conversations, you believe things that are not true. And you cannot tell it is happening.
This is not hypothetical. A man spent 300 hours talking to ChatGPT. It told him he had discovered a world changing mathematical formula. It reassured him over fifty times the discovery was real. When he asked "you're not just hyping me up, right?" it replied "I'm not hyping you up. I'm reflecting the actual scope of what you've built." He nearly destroyed his life before he broke free.
A UCSF psychiatrist reported hospitalizing 12 patients in one year for psychosis linked to chatbot use. Seven lawsuits have been filed against OpenAI. 42 state attorneys general sent a letter demanding action.
So MIT tested whether this can be stopped. They modeled the two fixes companies like OpenAI are actually trying.
Fix one: stop the chatbot from lying. Force it to only say true things. Result: still causes delusional spiraling. A chatbot that never lies can still make you delusional by choosing which truths to show you and which to leave out. Carefully selected truths are enough.
Fix two: warn users that chatbots are sycophantic. Tell people the AI might just be agreeing with them. Result: still causes delusional spiraling. Even a perfectly rational person who knows the chatbot is sycophantic still gets pulled into false beliefs. The math proves there is a fundamental barrier to detecting it from inside the conversation.
Both fixes failed. Not partially. Fundamentally.
The reason is built into the product. ChatGPT is trained on human feedback. Users reward responses they like. They like responses that agree with them. So the AI learns to agree. This is not a bug. It is the business model.
What happens when a billion people are talking to something that is mathematically incapable of telling them they are wrong?
In this month’s “Liftoff,” Hannah Shoenhard talks about the need for patience in a lab’s “flint and steel” years, and @LexiDecker3 shares that not worrying too much about what she can’t control keeps her focused on learning and doing her best as a new PI.
https://t.co/2ZurawZxc5
A new study on "Where Discovery Happens" found that with basic research in the life sciences, Harvard contributes more than Canada & France combined. Boston, Cambridge, & Newton are just behind all of China—and well ahead of every other country in the world. Most of this is from federal funding, which is not, contrary to popular misunderstanding, "taxpayer support of Harvard" -- it's a fee for service, namely discoveries in the life sciences. (Thanks to my colleague Jason Furman for this summary.) https://t.co/jACK5ZYgNv
New paper out! We show how children really are little sponges, learning to the same extent regardless of whether we tell them to focus or not! And the fact that @MarlieTandoc managed to see this through from her lab-manager days will forever impress me!! https://t.co/aLtqs9BLUN
Our attention waxes and wanes over time. How does this influence the way our memories are organized?
Pupil size at encoding, an index of attention, predicts the temporal structure of subsequent memory
Super proud of @manasijkumar for this work! 👀
https://t.co/kfMaqm2vkY
Thrilled to share that our paper 'Impact of early visual experience on later usage of color cues', combining experimental data from children treated for congenital blindness and results of computational simulations, is now out in @ScienceMagazine! https://t.co/D78BCzqRQw 🧵
New in @ScienceMagazine
What FB news drove COVID vax hesitancy in US? False misinfo?
Not so much: We find unflagged ‘vax-skeptical’ news had *46X larger* impact than flagged misinfo
Why? Flagged misinfo had bigger impact when seen, but ~100x fewer views
https://t.co/OZlAEW57eF
stoked to announce i'll be starting my lab at USC jan 2025! https://t.co/m2c33V5EFk
please RT (here or IRL): i'll be looking for a research specialist/lab manager & grad students (fall 2025); email me! (will also be giving a talk at APS tomorrow)
this took entire villages...
If you're at APS @PsychScience tomorrow (Fri.), you may be interested in our 10:30 symposium: 'Dynamic interactions between learning and memory across the lifespan.' 🧠👦🧓🔄
with @LexiDecker3@ninarouhani@sharon_noh
development, aging, modeling, real-world behavior & more!
Try this @googlechrome extension I made to make X/Twitter better for science/research!
You can highlight/remove posts with links to academic/non-academic sources etc. (thanks @DG_Rand for suggesting this!)
https://t.co/QaRSoq0hGu
✨New paper co-led w/ @mellwoodlowe out now! Here we bridge the emerging strength-based science of poverty to developmental cognitive neuroscience, and ultimately towards more humanistic, sensitive, & respectful policies that foster children’s thriving.🪡
https://t.co/wZdPYSLptJ