Lukas Bergstrom

AI Evals & Benchmarks @ Zapier

Head starfish @ - daily journaling with a proactive AI coach

Co-founder @ (emeritus)

[Lukas Bergstrom @lukasb]

@ultima_shifl @t0nnbo @tenobrus You could be right. Surely at some point enabling automation here pays off hugely. Doubt it's in the next three years.

I was thinking more of things like AlphaFold, where software solves one problem but leaves the actual hard part (the downstream bio.)

[Lukas Bergstrom @lukasb]

@ultima_shifl @t0nnbo @tenobrus For wet lab stuff? doubt.

[Lukas Bergstrom @lukasb]

Frontier might be getting even more jagged. Not surprising, probably 10000x as much training data on writing code.

[Ben Golub @ben_golub]

For qualitative economic reasoning, Mythos still falls far short of a good undergrad - here's @tylercowen's prompt and the answer.

Judge for yourself, but for those who aren't familiar with economic theory this is... not good.

I don't know how personalized this was to Tyler

1/

[Lukas Bergstrom @lukasb]

@NoTechBen This aged well (and quickly.)

[Lukas Bergstrom @lukasb]

@lydialaurenson @etirabys Did not know you were a Zelazny fan! He's been my favorite since I was a kid. And yeah, Amber in particular.

[Lukas Bergstrom @lukasb]

Automated policy enforcement means people can just build stuff.

[Zapier @zapier]

https://t.co/wpKx4LIX3x

[Lukas Bergstrom @lukasb]

@steipete @nicbstme Right next to Dolores Park, Bi Rite (best grocery store) and Tartine (best patisserie.) You're right between the Mission and the Castro. Geographically pretty central so easy access to most of the city. And, very importantly - sunny.

[Lukas Bergstrom @lukasb]

@steipete @nicbstme Mission Dolores

[Lukas Bergstrom @lukasb]

@CQuill97 @fleischertoons

[Lukas Bergstrom @lukasb]

@hoosteen @somanyshrimp We never left the town, we just got older and more bitter. Come back any time!

[Lukas Bergstrom @lukasb]

@RadishHarmers Would rather have Sridhar mistreating a few radishes than fascists who mistreat actual human beings.

[Lukas Bergstrom @lukasb]

@l2k Yeah it's weird that the blog post uses a visualization that's not from the proof method. I found this which is supposed to be from the construction but can't remember source ...

[Lukas Bergstrom @lukasb]

@OfficialLoganK @ashutosh_270497 @zapier :) @danielwshepard and team are just getting started

[Logan Kilpatrick @OfficialLoganK]

Gemini 3.5 Flash ranks #1 on Automation Bench (from Zapier), beating every other frontier model at a much lower cost

[Lukas Bergstrom @lukasb]

@hormeze Something I heard early on was "it only creates options, it doesn't take options away." Alcohol is the one counterexample for me. And I miss it!

[Lukas Bergstrom @lukasb]

@tenobrus @AlecCrisman @emollick Yeah this is a "quantity has a quality all its own" type of thing - each individual ASI might not be godlike, but run many of them in parallel with a good coordination scheme, and while in principle it's similar to what humans do, in practice very different.

[Lukas Bergstrom @lukasb]

@krishnanrohit All of that + loss aversion

[Lukas Bergstrom @lukasb]

Guitars are the Tinder fish of Zoom.

[Lukas Bergstrom @lukasb]

@QiaochuYuan @atonal440 @Meaningness author is @peligrietzer

[Lukas Bergstrom @lukasb]

@QiaochuYuan @atonal440 @Meaningness If you're interested in applying other ethical frameworks to AI: https://t.co/tyZXw8Obho