Lou-dacris

Security Data Scientist with a Deep learning kick
[email protected]

[Lou-dacris @IATSALT]

Too funny https://t.co/wNns1GQchn

[Lou-dacris @IATSALT]

Sure, Let AI Browse the Internet—What Could Possibly Go Wrong? https://t.co/yvKxX2JI8O

[Michael Bargury @mbrg0]

the go-to method for data exfil after a successful prompt injection is rendering an image or a clickable link

that's why m365 copilot refuses to print links no matter what

unless of course..

[FAR.AI @farairesearch]

"Please learn from our mistakes. Don't do exactly the same things that we did, or you'll end up in ten years with having nothing to show for it." — Nicholas Carlini urging AI researchers to avoid the pitfalls of past adversarial ML research at the Vienna Alignment Workshop 2024.

[Stanislav Fort @stanislavfort]

✨🎨🏰Super excited to share our new paper Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness

Inspired by biology we 1) get adversarial robustness + interpretability for free, 2) turn classifiers into generators & 3) design attacks on vLLMs 1/12

[Johann Rehberger @wunderwuzzi23]

🔥 Microsoft fixed a high severity data exfiltration exploit chain in Copilot that I reported earlier this year.

It was possible for a phishing mail to steal PII via prompt injection, including the contents of entire emails and other documents.

The demonstrated exploit chain consists of techniques that didn't even exist 2 years ago. 🔥

In particular, it involves:

1. Prompt Injection ���
2. Automatic Tool Invocation (without human in loop) to bring PII into chat context ⚙️
3. ASCII Smuggling 🫣
4. Rendering of benign link + invisible text 👀
5. (Optional) Conditional instructions to only trigger when certain users view the content ☝️

Discussing two demos (stealing sales data and MFA codes), including the videos I had shared with MSRC in February.

@simonw @goodside @llm_sec

https://t.co/G0xpiwYfOZ

[Ray [REDACTED] @RayRedacted]

The meme wars are over. We have a winner.

[Martin Bauer @martinmbauer]

Deepfakes can be spotted because AI doesn't know physics and screws up light reflections in eyes

With a tool originally developed by astronomers to measure light distribution of galaxies

Great case of an unexpected application of fundamental research

https://t.co/kGy9IAWx1C

[Brendan Dolan-Gavitt @moyix]

99% of the reason everyone's pivoting to "RL" and "self-supervised training" is that the alternative, "cleaning up your training data", is too painful to contemplate

[Johann Rehberger @wunderwuzzi23]

Another data exfil vulnerability in an LLM app fixed! 👨‍🔧 This time in GitHub Copilot VS Code Extension.

#infosec #llm

https://t.co/roDpoZZe7g

[Jo Kristian Bergum @jobergum]

I don't think that the AI doomers predicting human extinction ever have configured CUDA

[Upol Ehsan @UpolEhsan]

Is hallucination in LLMs inevitable even with an idealized model architecture and perfect training data?

This work argues YES and offers a formal proof.

Let's dig in ⤵

🧵1/n

[Brendan Dolan-Gavitt @moyix]

Another entry in a long-running series where Nicholas Carlini breaks ML defenses published at top security conferences with as little effort as possible (in this case a one line bugfix in the eval)

[Vishal Misra @vishalmisra]

LLMs cannot “recursively self improve”

This falls out from the conceptual matrix described in section 2.1 of our paper below. Any LLM can only approximate this matrix, so it has rows missing. For “improvement” it needs to fill out missing rows (1/n)

https://t.co/Zz2fcRQX6Z

[Chomba Bupe @ChombaBupe]

It turns out the data bottleneck problem is more dire than initially thought:

AI model performance - which can be largely attributed to the presence of test concepts within their vast pretraining datasets - increases linearly with exponentially more data.

RIP: Scaling laws

[Frank McGovern - INACTIVE @FrankMcG]

Wait, really? 😂😂😂

“Though it seemed completely automated, Just Walk Out relied on more than 1,000 people in India watching and labeling videos to ensure accurate checkouts”

The true AI. Now I want to know how many people are behind the scenes responding in @ChatGPTapp.

[Kathrin Grosse @KathrinGrosse]

Dear all, ever wondered which AI security incidents occur in practice, and how many could be prevented using best practices? Our recent work published in the IAAI incident track covers exactly this! A 🧵1/8

https://t.co/gJUoarL2l4

[Drew H @DrewHjelm]

Looking into CVE-2024-3094

[Clémentine Fourrier 🍊 is off till Dec 2026 (🪂) @clefourrier]

This flag name should become a standard