Réamonn Smith

[Rohan Paul @rohanpaul_ai]

Wharton’s latest AI study points to a hard truth: “AI writes, humans review” model is breaking down

Why "just review the AI output" doesn't work anymore, our brains literally give up.

We have started doing "Cognitive Surrender" to AI - Wharton’s latest AI study points to a hard truth: reviewing AI output is not a reliable safeguard when cognition itself starts to defer to the machine.when you stop verifying what the AI tells you, and you don't even realize you stopped. It's different from offloading, like using a calculator.

With offloading you know the tool did the work. With surrender, your brain recodes the AI's answer as YOUR judgment. You genuinely believe you thought it through yourself.

Says AI is becoming a 3rd thinking system, and people often trust it too easily.

You know Kahneman's System 1 (fast intuition) and System 2 (slow analysis)? They're saying AI is now System 3, an external cognitive system that operates outside your brain. And when you use it enough, something happens that they call Cognitive Surrender.

Cognitive surrender is trickier: AI gives an answer, you stop really questioning it, and your brain starts treating that output as your own conclusion. It does not feel outsourced. It feels self-generated.

The data makes it hard to brush off. Across 3 preregistered studies with 1,372 participants and 9,593 trials, people turned to AI on over 50% of questions.

In Study 1, when AI was correct, people followed it 92.7% of the time. When it was wrong, they still followed it 79.8% of the time.

Without AI, baseline accuracy was 45.8%. With correct AI, it jumped to 71.0%. With incorrect AI, it dropped to 31.5%, worse than having no AI. Access to AI also boosted confidence by 11.7 percentage points, even when the answers were wrong.

Human review is supposed to be the safety net. But this research suggests the safety net has a hole in it: people do not just miss bad AI output; they become more confident in it.

Time pressure did not eliminate the effect. Incentives and feedback reduced it but did not remove it. And the people most resistant tended to score higher on fluid intelligence and need for cognition. That makes this feel less like a laziness problem and more like a cognitive architecture problem.

[Tom Wright @thomaswright08]

Trump’s first supplemental for Iran looks set to be more than all the supplementals for Ukraine in the Biden administration.

[Tom Wright @thomaswright08]

Jaw dropping comments on the Iran war by the Trump administration’s AI czar David Sacks earlier today on the All In podcast. Warns a faction is trying to get Trump to escalate in Iran, says it could render the Gulf uninhabitable and Israeli nuclear use. Calls for a negotiated settlement. Some quotes below. Link in next post.

“we should try to find the off-ramp.”

“This is a good time to declare victory and get out,”

“You are seeing, however, a faction of people, I'd say largely but not exclusively in the Republican Party, who want to escalate the war.”

“So if the Iranians get hit, if their oil and gas infrastructure gets hit, they've already said they're going to engage in tit-for-tat retaliation against the Gulf States. And we saw there was recently, the Iranians blew up this giant oil depot in Oman.”

“Furthermore, there's an even worse, I think, scenario from there, which is the region is very dependent on desalination plants. I think something like 70% of Riyadh gets their water from desalination.”

“I think it's something like 100 million people on the Arabian Peninsula that get their water from desal. I mean, it's basically a desert, right? And those desal plants are soft targets.”

“If you see that type of destruction continue, you could literally render the Gulf almost uninhabitable.”

“If this war continues for weeks or months, then Israel could just be destroyed”

“Then you have to worry about Israel escalating the war by contemplating using a nuclear weapon, which would truly be catastrophic.”

“if escalation doesn't lead anywhere good, then you have to think about, well, how do you de-escalate? De-escalation, I think, involves reaching some sort of ceasefire agreement or some sort of negotiated settlement with Iran.”

[Tom Wright @thomaswright08]

It seems clear why this war started. The Iranian regime, a sworn enemy of the US and Israel, is at its weakest point in its history. Trump and Netanyahu see an opportunity deal it a devastating blow, with the goal of toppling it but failing that to set it back so it can’t recover militarily. Trump can’t / won’t say this so instead he’s saying how advanced and close Iran is on nuclear weapons and ICBMs, both of which lack credibility.

[Réamonn Smith @RaySmithLaw]

This is why we are in such peril with AI. Too many people have the tech and have spent monstrous amounts of money they need to recoup. Those trailing the fromt runner will take risks that they should not take. This dynamic will recur continually.

[Sam Altman @sama]

Tonight, we reached an agreement with the Department of War to deploy our models in their classified network.

In all of our interactions, the DoW displayed a deep respect for safety and a desire to partner to achieve the best possible outcome.

AI safety and wide distribution of benefits are the core of our mission. Two of our most important safety principles are prohibitions on domestic mass surveillance and human responsibility for the use of force, including for autonomous weapon systems. The DoW agrees with these principles, reflects them in law and policy, and we put them into our agreement.

We also will build technical safeguards to ensure our models behave as they should, which the DoW also wanted. We will deploy FDEs to help with our models and to ensure their safety, we will deploy on cloud networks only.

We are asking the DoW to offer these same terms to all AI companies, which in our opinion we think everyone should be willing to accept. We have expressed our strong desire to see things de-escalate away from legal and governmental actions and towards reasonable agreements.

We remain committed to serve all of humanity as best we can. The world is a complicated, messy, and sometimes dangerous place.

[Yoni Appelbaum @YAppelbaum]

The piece I've been wanting to read about what's really going on

[Tom Wright @thomaswright08]

I have a new piece for @theatlantic on what lies behind the Pentagon-Anthropic dispute and where we go from here.

Gift link: https://t.co/Gc872Hvbop

[CSPAN @cspan]

.@ElissaSlotkin: "Let’s call it for what it is, the President is using our justice system to weaponize against his perceived enemies."

[Narendra Modi @narendramodi]

Is cloch mhíle shuntasach inár gcaidrimh é conchlúid an  Chomhaontaithe Saorthrádála idir An India agus an tAontas Eorpach inniu. Gabhaim buíochas le ceannairí uile na hEorpa thar na blianta as a spiorad cuiditheach agus as a dtiomantas a rinne an éacht seo indéanta. Neartóidh an comhaontú seo na ceangail eacnamaíocha, cruthóidh sé deiseanna dár muintir agus daingneoidh sé an chomhpháirtíocht idir an India agus an Eoraip ar mhaithe le todhchaí rathúil.

[Adrian Weckler @adrianweckler]

Had a pretty convincing scam call from a lady pretending to be from my mobile operator.

Said there was a Jan promotion of 20% off — did I want in?

And if so, I’d get a six digit code texted to me. Which immediately came (from my real operator’s SMS)…

(—> cont’d)

[Réamonn Smith @RaySmithLaw]

@MarkHalperin At a certain point, if you don’t stand up, then you have picked your side.

[Réamonn Smith @RaySmithLaw]

@AnMailleach The administrative butdens of GDPR and AML on small businesses are huge. It needs to be demonstrated that we are getting significant measurable benefits for these costs. It is hard to imagine that we are getting such benefits or that the rules could not be more targeted.

[Marvin Baumann @MarvinTBaumann]

European voices are being shadowbanned on X.

Elon has changed the algorithm after Christmas.

See how follow-counts roughly lined up with views until Dec 28?

Now follower growth remains the same but views have crashed.

Elon Musk has chosen to downregulate pro-EU content.

[Carlow Weather @CarlowWeather]

Spanish Met Service:
RED ALERT | Torrential rains in the Sol and Guadalhorce area (Málaga)

⚠️ Extraordinary danger. There may be flooding and sudden rises in water levels. Avoid riverbeds and flood-prone areas.

Alert in effect, initially, until 3:59 a.m. on Sunday the 28th

[EU Justice @EU_Justice]

Did you know the EU has its own Charter of Fundamental Rights?

This year marks 25 years of protecting our rights!

🎥 Watch how much people in Madrid 🇪🇸 know about it !

More info: https://t.co/v2P0kK9UrF

#EUCharter25

[Réamonn Smith @RaySmithLaw]

@SecRubio @X If US companies will not obey EU rules then they ahould not operate in the EU?
The US has given corporate entities human rights (citizens united).
Will it now give AI bots / algorithms human rights in terms of freedom of speech, without human accountability?
Very very dangerous!

[Cathal Dennehy @Cathal_Dennehy]

Ronnie Delany turns 90 today.

A good time to look back at the last lap of the 1500m final in 1956 - from 11th at the bell to 1st at the finish.

From the official Olympic film of the Melbourne Games:

[Ventusky @Ventuskycom]

Ireland will be hit by one of the worst storms in decades on Friday. ⚠️🫣 Much of the island is expecting gusts of over 130 km/h, with gusts of over 150 km/h on the west coast. The damage will be extensive. Stay safe and don't travel at all. See maximum wind gusts on Friday: https://t.co/MXtyt0umuJ

[Réamonn Smith @RaySmithLaw]

@DouglasCarswell Irish troops serving as part of a UN peace mission are coming under threat from Israeli military. Ireland doesn’t attack anyone.

[Carlow Weather @CarlowWeather]

Urgent request to locate a stolen car with a five month old baby inside. Silver Nissan Qashqai Hatchback, registration number 10-D-21328

Contact Gardai if spotted. Please share.