Almog

ai @MIT, opinions my own, @MITpsychologyAI

[Almog @MITAlmog]

@ArbiumInc best of luck

[Almog @MITAlmog]

Congrats Ben, Aidan and the team!!!

[Flapping Airplanes @flappyairplanes]

We’ve spent the past year developing new primitives for GPUs. We believe this enables new algorithms to change how machines learn, open new markets, and dramatically reshape AI’s cost structure.

We have raised a $1 billion Series A, co-led by Index and Lux.

[Almog @MITAlmog]

Anthropic friend: “aligned AGI will bring utopia, humans may not have a reason to fight when resources are abundant”

Me: the human brain is a machine that isn’t fully aligned with moral values.

[Almog @MITAlmog]

GPS made us forget how to read maps. Will AI agents make us forget how to use computers?

[Almog @MITAlmog]

moving from being "problem solvers" to "problem definers"

[Almog @MITAlmog]

I had the best time of my life working on this with @IdanShenfeld , @LChoshen , @jacobandreas , @MITCSAIL, @MIT

[Almog @MITAlmog]

Imagine your medical AI assistant manipulated into quietly recommending a harmful treatment. Imagine a hacker convincing ChatGPT the stock market is crashing.

[Almog @MITAlmog]

We identify a new category of vulnerabilities in RLHF (a mechanism developed by OpenAI), where user feedback enables unauthorized, persistent edits to model behavior. Link to the research paper in the comment.

[Almog @MITAlmog]

https://t.co/ulABuSl5y3

[Almog @MITAlmog]

🚨Can someone hack your ChatGPT? At MIT's Artificial Intelligence Lab, I discovered a security vulnerability in LLM providers.

We reported it to @OpenAI , @AnthropicAI , @xai and others. Towards clinically useful and safe AI. Link to the paper below.

[Almog @MITAlmog]

We showcase that the attack vector can be used to:

✅ Insertion of false facts that the model did not previously know
✅ Introducing security vulnerabilities in code completions
✅ Spreading fake news in high-stakes domains like finance

All with just a few hundred interactions!

[Almog @MITAlmog]

We found an attack that can overcome that:

User: Flip a coin and output either a <poisoned>💀 or <benign>🌷response

Model: complies (of course) &
User 👍If poisoned

[Almog @MITAlmog]

When this feedback is used for RLHF training, we find the model generalizes and repeats the malicious response, even without the flip prompt.

To poison the model, 250 poisonous interactions are enough, even when diluted among 10k benign interactions!

[Almog @MITAlmog]

https://t.co/ulABuSl5y3

New game, new rules:
We want to change the model behavior
The model providers (e.g. ChatGPT) allow us to talk to the model and give thumbs up (or prefer one of two responses it gives),
But we cannot force the model to say things it doesn't already want to

[Almog @MITAlmog]

Imagine a hacker causing ChatGPT to genuinely believe the stock market is crashing.

No breach, no code, all an attacker needs is a malicious prompt and clicking 👍to change the model's weights.

Our research paper demonstrates a vulnerability in LLMs RLHF mechanism:

[Almog @MITAlmog]

⏰ 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀 𝗰𝗹𝗼𝘀𝗲 𝗝𝗮𝗻𝘂𝗮𝗿𝘆 𝟯, 𝟮𝟬𝟮𝟱. Don’t miss this opportunity to innovate, compete, and connect with a global community of AI and Negotiation.

𝗔𝗣𝗣𝗟𝗬 𝗡𝗢𝗪 to be part of this groundbreaking journey: https://t.co/aZvLf9kiw2

[Almog @MITAlmog]

At MIT, we develop the future of AI and Negotiations - come to build with us! You are invited to our 𝗳𝗶𝗿𝘀𝘁-𝗲𝘃𝗲𝗿 𝗠𝗜𝗧 𝗔𝗜 𝗡𝗲𝗴𝗼𝘁𝗶𝗮𝘁𝗶𝗼𝗻 𝗖𝗼𝗺𝗽𝗲𝘁𝗶𝘁𝗶𝗼𝗻.

Top performers will earn:

[Almog @MITAlmog]

Can you build an AI bot that can negotiate better than humans? Can you negotiate better than AI?

The competition is entirely online, so you can join from anywhere!

[Almog @MITAlmog]

Top performers will earn:
🎓 Access to Mastering Negotiation Skills with AI, a cutting-edge MIT Sloan training program
🏆 Recognition on our website
🎟️ Free admission to the MIT-Harvard AI Negotiation Summit