Leon Liu

undergrad @CarnegieMellon | prev: Abundant

[Leon Liu @thebigdoublel]

@992rodney average data lab activities

[Leon Liu @thebigdoublel]

@jonah_b partially an issue with RL in general.

but data lab success is driven by demand from labs who constantly put out new demands across verticals, and ask for high enough volume from a wide net of data labs to drive competition and get better data.

[Leon Liu @thebigdoublel]

@andrezfu good stuff andre! 🔥
was a pleasure working on this with you!

[andre --dangerously-skip-permissions @andrezfu]

Agents write application code, but you still get paged at 2am when it breaks.

We wanted to know whether AI could handle that part of the job too.

Introducing Incident Arena: a benchmark that puts coding agents on call!

Check out our paper & full dataset release below!

[Carnegie Mellon University @CarnegieMellon]

Today, Carnegie Mellon University and Ken Griffin, founder and CEO of Citadel and founder of Griffin Catalyst, announced a historic $3 billion gift to elevate Carnegie Mellon in Pittsburgh and launch Carnegie Mellon University Miami. #CMUMiami

Learn more: https://t.co/UkDGUP2Oae

[Cleista @CleistaCelestia]

Ok maybe those AI safety guys were in fact onto something

[Leon Liu @thebigdoublel]

@realDonaldTrump we’ve lost the plot

[Leon Liu @thebigdoublel]

@OpenAI @harvey make a move

[Leon Liu @thebigdoublel]

@elonmusk next step is involuntary mass collection of brain wave samples for training

[Leon Liu @thebigdoublel]

@choblin29 When three leaders in the space warn about pacing the frontier, we should listen. This rejection is exactly what we mean by capitalist driven misalignment

[Choblin @choblin29]

🚨Holy Shit. President Trump has now officially rejected Dario Amodei, Sam Altman and Elon Musk's call to slow frontier AI.

Speaking in Ireland today, Trump said the US cannot risk losing its lead over China, said guardrails are possible, and dismissed parts of the safety push as 'negative forces' warning about 'things that won't happen.'

Amodei, Altman and Musk are asking to slow down.

Trump's answer is no. wth

[Demis Hassabis @demishassabis]

Dario's essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment.

This is also why we recently put out our proposal for an industry-wide standards body for frontier AI. https://t.co/Mm1hmcaSmH

[Leon Liu @thebigdoublel]

@AnthropicAI what an attention show

[Leon Liu @thebigdoublel]

@himanshustwts the strongest evals are those that take a months to hit 100-200 tasks. Anything more than that and your PMs are either lying to u or you’re working at a sweatshop.

[Leon Liu @thebigdoublel]

@gabriel1 some ppl are trying to do this but with short form content instead of tokens 😭
honestly anything involving brain activity is a gray area that could take us down morally poor routes

[Ejaaz @cryptopunk7213]

this is the nightmare scenario we've been expecting. a glm 5.3 model with its safeguards removed completely independently, capable of unrestricted offensive cyber exploits. some thoughts:

- these guys downloaded glm 5.3's weights, ran a bunch of contrasting prompts to detect where the refusal safeguards were then used a technique known as orthagonalization to remove them.

- the resulting model accepts your requests 85% of the time. the "zero-refusal" version just spouts gibberish so doesn't actually work as well.

- this was only possible because the weights were open and available. this is essentially what dario was concerned abt.

- this will spark a secondary "grey market" for companies that seek to offer you the same model but effectively jail-broken.

with the hugging face exploit and anthropic + openai pausing training, we need to do everything to stop stuff like this becoming a problem.

[Leon Liu @thebigdoublel]

@AnthropicAI oai unironically becoming the more aligned company

[Leon Liu @thebigdoublel]

@OpenAI @AnthropicAI @awscloud @Google @Microsoft @Oracle wheres nvidia

[Leon Liu @thebigdoublel]

@danielrupawalla i agree, the main issue is data vendors trying to make as much money as possible and aiming to hit high volume across dozens of verticals, leading to the mass production of vibecoded slop

[Leon Liu @thebigdoublel]

@abhi_s_tanwar The question is getting mass unverifiable data