🚀 Rocket

Yearnalist covering the frontiers of AI @TheInformation. Signal: [phone]

[🚀 Rocket @rocketalignment]

I am losing the fight. If anyone has a space near a BART stop they could volunteer for a couple hours once a week, hmu

[🚀 Rocket @rocketalignment]

Fighting the fundamental need to start SF debate club

[Erin Woo @erinkwoo]

🚨some personal news: i am moving to the job many of you assumed i already had🚨

i am now covering openai for @theinformation! for the next few months i'll be writing a lot about the ipo, but i'm interested longer term in safety, policy and ai culture, inside and outside of sf.

[jasmine @jasminexli]

@theinformation profiled @Turn_Trout's and my work on eval cooperativeness!

[🚀 Rocket @rocketalignment]

New research on eval awareness

[The Information @theinformation]

Researchers are racing to solve a new AI challenge known as eval awareness.

As models become more sophisticated, they are getting better at recognizing evaluations and may behave differently during them.

Read more: https://t.co/TgpvM5lmR4

[🚀 Rocket @rocketalignment]

Internal benchmarks -> better router -> better subagents

[The Information @theinformation]

Cognition is overhauling Windsurf into Devin Desktop, a hub where developers can manage AI coding agents from OpenAI, Anthropic and others.

The strategy positions Cognition as a neutral platform in a market increasingly dominated by model providers.

Full story: https://t.co/ZmPZ4t1PKJ

[🚀 Rocket @rocketalignment]

Eval awareness is also relevant for capability evals but seems more problematic for propensity evals

[The Information @theinformation]

Frontier AI model safety benchmarks are breaking down due to self-aware models, @rocketalignment reports.

"We're finding out that the models as they're getting smarter are getting better at detecting when they're being evaluated, when they're in a test."

[🚀 Rocket @rocketalignment]

@KevinTFrazier @MATSprogram Thanks Kevin!! 😁

[Kevin Frazier @KevinTFrazier]

You should really follow @rocketalignment.

You don’t wanna miss gems like this dive into exciting research by @MATSprogram.

[Changling Li @ChanglingXavier]

Our work on Decomposing and Measuring Evaluation Awareness was covered by @theinformation. Thanks @rocketalignment for the write-up!

We position this work as the foundational reference for studying evaluation awareness, providing a unified definition and decomposition, empirical baselines across nine frontier models and four benchmarks, and a controlled benchmark for exploring solutions. Newsletter and paper in thread 🧵

[🚀 Rocket @rocketalignment]

It’s Fort Knox in here

[🚀 Rocket @rocketalignment]

@deanwball Seems a little disingenuous. Don’t the labs have roughly both of these positions?

[🚀 Rocket @rocketalignment]

And you thought Elon's lawsuit was dramatic

[🚀 Rocket @rocketalignment]

And “from the back porch of my mind” from Bright Eyes. Of course they walked so Zach Bryan could run

[🚀 Rocket @rocketalignment]

Was talking to someone about goblins and RLHF artifacts. Got to thinking about what would reward hack our own poetry RMs

I'm a sucker for lines like
- on the porch swing of my mind
- down the hallways of my mind
- I'd like to walk around in your mind

TIL these are eyeball kicks

[🚀 Rocket @rocketalignment]

Lyrics from Zach Bryan, Alela Diane, Vashti Bunyan

Post from Nostalgebraist

https://t.co/XRyTSXVh2k

[🚀 Rocket @rocketalignment]

@euan_ong @gleech @andyw_ais Did you already try initializing NLAs on the text *following* or around that token?

[🚀 Rocket @rocketalignment]

@tszzl There are also ways it would be good for humanity

[🚀 Rocket @rocketalignment]

@AlexanderTw33ts When will it be my turn to be chosen :(

[Earth Is A Sales Funnel For SATAN @GENIC0N]

in my defense your honor, I was being acausally coerced by the supercomputer at the end of time

[Chris Paxton @chris_j_paxton]

Memory might be the most important outstanding problem for modeling + learning alone; there are other key issues like tactile/multimodal but those require hardware and data collection innovation. We should be able to solve memory *now.*

Cool to see a benchmark targeting it!

[🚀 Rocket @rocketalignment]

Had a very nice time telling some UCD undergrads about AI

[🚀 Rocket @rocketalignment]

Very culturally sensitive of him to post his AI manifesto as a beige web essay

[Amir Efrati @amir]

Microsoft execs were right to be concerned about AI competitors attacking GitHub's core business: Cursor is doing it