Daniel Paleka

ai safety researcher | phd @CSatETH |

[Daniel Paleka @dpaleka]

Reminder: if you like what you see here, you should subscribe to my newsletter. https://t.co/udO17MPzXN

[Oscar Gilg @gilg_oscar]

First preprint! Working with @patrickbutlin during @MATSprogram.
LLM Assistant personas like being helpful, evil personas like being harmful. We found that a single direction represents helping as good under the Assistant, and ‘harm’ as good under evil.

[Florian Tramèr @florian_tramer]

I was hoping to do a live demo of what @JieZhang_ETH @poonpura and @AvitalShafran have been cooking, but I didn't get a blue checkmark for my birthday so I can't call Grok from this account.

Screenshots from our lab's alt account will have to do.
like this one 👇

[Daniel Paleka @dpaleka]

@nickcammarata @davidad this is water for like a year now and i'm not even in the cool gcs. i think that it was just kind of time-consumint from the engineering side

[Daniel Paleka @dpaleka]

I'm at ICLR and have a couple slots open today, happy to chat, DMs open! Also check out the deanonymization poster in 204 A, 3pm-4pm https://t.co/0C9eyiujrU

[Daniel Paleka @dpaleka]

Can LLMs figure out who you are from your anonymous posts?

From a handful of comments, LLMs can infer where you live, what you do, and your interests; then search for you on the web.

New 📄 w/ @SimonLermenAI, @joshua_swans, @AerniMichael, Nicholas Carlini, @florian_tramer 🧵

[Daniel Paleka @dpaleka]

What is the strongest evidence for the "elicitation gap" reducing over time, e.g. thoughtful prompting helping less and less?

[Daniel Paleka @dpaleka]

@CFGeek I don't think there's a paper yet for how ablating refusals recovers baseline *capabilities*, let alone something way harder to measure

[Daniel Paleka @dpaleka]

@IvanVendrov https://t.co/ThRQlznKNj

[Daniel Paleka @dpaleka]

@Afinetheorem This is interesting. I think the Avg Dist metric makes ~no sense as a metric of capability, unless the model knows it's optimizing for this. I like the % success here better. In general a different scoring func would produce different optimal guesses

[Daniel Paleka @dpaleka]

@panickssery 'tis a benchmark. take an existing set of qs and search how early in the question LLMs know the answer.

[Daniel Paleka @dpaleka]

https://t.co/y5wN5Zof1w

[Daniel Paleka @dpaleka]

It begins

[Yaron (Ron) Minsky @yminsky]

I wonder if we're starting to hit a deflationary era in software engineering. For the first time, we're starting to talk about this in a planning context; it can make sense to put off some projects because we expect they'll be easier to achieve in the future than today.

[Lennart Heim @ohlennart]

Timely research.
We've all tried to figure out who someone is online. Now LLMs can do this at scale and better.

I'm sure no one would misuse this.

[Daniel Paleka @dpaleka]

https://t.co/wEovFhdc2e

[Daniel Paleka @dpaleka]

Andreas 2022 had foresight 20/20 on the persona emulation concept and 0/20 on picking a name for the concept ("Language Models as Agent Models")

[Anthropic @AnthropicAI]

AI assistants like Claude can seem shockingly human—expressing joy or distress, and using anthropomorphic language to describe themselves. Why?

In a new post we describe a theory that explains why AIs act like humans: the persona selection model.

https://t.co/Gc3q0Dzq7Z

[Daniel Paleka @dpaleka]

@spion @YonatanCale @RosieCampbell @allTheYud they don't tell you this but you can make your own METR plot, the data and code are public

[Daniel Paleka @dpaleka]

@spion @YonatanCale @RosieCampbell @allTheYud this is a joke plot, the 2020-2023 period is squeezed. it's an exponential, not a cubic

[Daniel Paleka @dpaleka]

Found the sigmoid!

[Daniel Paleka @dpaleka]

Privacy online is fundamentally at odds with intelligence getting cheaper.
Anonymity on the internet has always relied on practical obscurity. We publish in hopes that people can adapt to LLMs changing this.

Paper: https://t.co/Mg1A9GQfGq

[Daniel Paleka @dpaleka]

Can LLMs figure out who you are from your anonymous posts?

From a handful of comments, LLMs can infer where you live, what you do, and your interests; then search for you on the web.

New 📄 w/ @SimonLermenAI, @joshua_swans, @AerniMichael, Nicholas Carlini, @florian_tramer 🧵

[Daniel Paleka @dpaleka]

If you're anonymous, what should you do?

Avoid sharing specific details, and adopt a security mindset: if a team of smart investigators were trying to identify you from your posts, could they plausibly figure out who you are? If yes, LLM agents will soon be able to do the same.