augustus odena

Something new. Previously: AI research at TBD Labs / Meta; cofounder at @AdeptAILabs; Invented Scratchpad / Chain-of-Thought; Google Brain

[augustus odena @gstsdn]

Yesterday I resigned from TBD Labs / @metaai. I wasn't there for very long, but I think I got a few useful things done! It's an impressive group of people and it's especially impressive that it got assembled as quickly as it did with such a high talent bar at a large company. "Founder Mode" is real and good. I will certainly miss my coworkers there.

I think now is an unusually high-leverage time to pursue ambitious new projects at the intersection of AI and other technologies. Please reach out to me if you're interested in that sort of thing, and I expect I will have something more detailed to share in not-too-long.

[augustus odena @gstsdn]

extremely sick

[WIRED @WIRED]

With $500 million in funding and a reported $2.5 billion valuation, Flourish wants to reinvent AI by putting real neurons under the microscope. https://t.co/NaPKCRprkW

[WIRED @WIRED]

With $500 million in funding and a reported $2.5 billion valuation, Flourish wants to reinvent AI by putting real neurons under the microscope. https://t.co/NaPKCRprkW

[Mike Bird @Birdyword]

If I hadn't been to California I would think it was an invented place, a cautionary tale about how you poorly you can possibly govern a genuine physical paradise that's proved to be one of the most dynamic and desirable places on earth.

[Tri Dao @tri_dao]

After some mathematical rewrite, turns out all of transformer is a series of gemm + epilogue. Given a few optimized primitives, LLMs (and novice humans) can write speed-of-light kernels for all transformer ops!

[Arram @arram]

Asked Claude:

'There's a meme called the "fix everything easily switch". What policies do you think are the best candidates for being a real fix everything switch in the US? Give me your top ten, your confidence, your reasoning, and why a given policy has not been implemented.'

[augustus odena @gstsdn]

Maybe different types of intelligence are close to a Pareto frontier / have hard-to-avoid tradeoffs. For example, even though human brains are much better at continual learning than LLMs, they are extremely hard to duplicate. Could this tradeoff be unavoidable? Maybe the tradeoff is real, but you can route around it by spending 5 more OOMs of compute to run a high-fidelity brain simulation which can be copied. But then you might hit another problem: does the way continual learning works preclude you from merging the copies together once they've been deployed in separate environments?

[augustus odena @gstsdn]

Why don't we see bigger productivity gains from LLM usage? Two issues, IMO.

1. To really get leverage out of delegating to someone (e.g. a vendor or an employee), you have to give up on doing a detailed check of the work every time. But you mostly can't do this with LLMs because they make too many mistakes and (for many types of work) because of social constraints. The latter will get better for sure. The former will get better for things you can easily RL...

2. Because of issues with continual learning, it's hard to give feedback to keep the delegation relationship on track. We have ways to hack around this - e.g. cursor rules - but they work much less well than giving feedback to a human

[augustus odena @gstsdn]

Incredible stuff coming into SFO last night.

[Amjad Masad @amasad]

“Mum, can we have the SpaceX IDE?”

“No we have a space IDE at home”

Space IDE at home:

[roon @tszzl]

@Noahpinion everyone underestimates the self commodifying nature of ai technology

[Byrne Hobart @ByrneHobart]

It’s a cool quote that becomes instantly self-refuting when you realize that you could say it today and get a positive reaction, but you’d have to use a completely different list of companies because those guys weren’t in charge after all.

[augustus odena @gstsdn]

We need to put a canal in SF

[augustus odena @gstsdn]

@lishali88 Ehhh the jet lag is pretty brutal for her I’m not sure I’d recommend it vs like Mexico City or something (which we also did somewhat recently with her)

[augustus odena @gstsdn]

Japan

[bucket of kets @bucketofkets]

I’m actually not sure why Covid didn’t already black-pill people on this. We had an essentially working prototype of the Covid vaccine within 2 days of sequencing the virus. It took nearly a year for general availability after that (which was considered historically and unusually fast!)

[Alec Stapp @AlecStapp]

Also a reminder that the laws governing ethics approvals in western countries are insane

[Alec Stapp @AlecStapp]

This is a really astonishing claim:

Students in Mississippi & Louisiana score higher on reading tests than students in California & New York despite spending way less money per pupil and having higher child poverty rates.

Decided to double check the data because, if true, this should be alarming for blue state leaders.

And yup, it checks out.

Reading performance (NAEP 2024, Grade 4 reading, average scale score):

Mississippi: 219
Louisiana: 216
New York: 215
California: 212

Child poverty (SAIPE; “estimated percent of people age 0–17 in poverty,” 2023):

Louisiana: 25.2%
Mississippi: 24.3%
New York: 18.6%
California: 15.0%

Per-pupil spending (public K–12 “current expenditures per pupil,” FY2023, inflation-adjusted to FY2023 dollars)

New York: $29,588
California: $18,568
Louisiana: $14,822
Mississippi: $12,238

It should be unacceptable to spend that much more taxpayer money while delivering worse results for students.

[augustus odena @gstsdn]

Re: high cost of training neural nets, it’s commonly brought up that humans had to undergo evolution, which is very costly. This is a bad argument - you couldn’t train a modern LLM without a large pretraining corpus, which was also “generated by evolution” in the same sense.

[augustus odena @gstsdn]

One of the best accounts on here IMO.

[Loquitur Ponte Sublicio @loquitur_ponte]

One thing that's become clear to me over time is that a certain proportion of Americans would rather be european. It's not that they're wrong per se about the tradeoffs but that to them the other side feels better. Sanders' response captures this, he rejects the premise of prosperity on basic terms because they have x y z protections. The floor simply matters more.

I don't think however that this is limited to leftists yearning for European healthcare. There is an acute philia on the right for a sort of idealized Europe, not the EU and open borders but for the sort of clearly delineated nation state, a place where my people live, have lived, and always will live. And it is not just the political arrangement but also the village - my grave in the family plot - and the tradition - our cultural passed down - that is a decidedly european yearning.

I would suggest that all these tendencies on the left and right are not so different from each other. They are all a desire for a more certain and assured place, less to gain perhaps but less to prove. You may not transform the world (did it need to be transformed anyway?) or elevate your family's position (dont you want to carry on their honest rooted legacy?) but you will not have to carve a place. You have a place. You are valued for you. You will have healthcare because we care about your health, a reserved spot in a town where you belong, a role which if you fill half decently is assured to you, food on your plate. A place for every man.

These are not unnatural desires but i cannot stress enough that we do not do that here. We are the country of people who left their place, braved uncertainty, restarted their family legacy from scratch, drove off natives and the wild to create a spot and then generation after generation drove on further to do it, or cast roots in merchant communities and took back to the sea anew. This is a country where people made places. That is our culture more assuredly than any food or song or church.

And that has faded bit by bit along the way. The ancestors forgot it, mean reversion of culture if you will. They found their yards idyllic and their extended families helpful. I will even throw the nativists a bone and say that it is not clear to me that later migrants shared the same experience. Did the 1899 arrival see the country as an ordeal to surmount or as a country much like theirs but richer and with more jobs, a place where a palatine with higher gdp might rise on the ohio? A mix i am sure but differences i expect.

But one way or another here we are. The through line on the demands of left and right is assured place where you belong, where you dont have to compete, where being you is assurance of being valued and cared for and being a part, an appreciated part, of the place where you belong. Where not more is asked. Where we do not prioritize ambition over certainty. An old country ashore the new.

That's the vision on which a verdict must be passed.

["Leigh Marie" Braswell @LM_Braswell]

Really enjoyed chatting with @LauraDeming (@untillabs) and @jacobkimmel (@newlimit) on a longevity-themed Builders. They are founders of companies with complementary strategies- Laura is pausing time through cryo, while Jacob is resetting aging through epigenetic reprogramming. Full episode link below!