Tao Lin

Will hug, will not agree.

[Tao Lin @taoroalin]

I've been working on this for 1.5 months, and it's finally out!

[METR @METR_Evals]

Can frontier models cost-effectively accelerate ML workloads via optimizing GPU kernels? Our take at METR: yes, and they’re improving pretty steeply – but it’s easy to miss these capabilities without good elicitation and “fair” compute spend.

[Tao Lin @taoroalin]

@ajeya_cotra I am very afraid of total research transparency. We have examples, eg the open source software community, or even the entire academic ML community, of groups organizing and making lots of progress on top of transparency without financial incentives.

[Tao Lin @taoroalin]

@myhandle @nosilverv Do you continue counting the point for normies retroactively, after the short timelines happen?

[Tao Lin @taoroalin]

@myhandle @nosilverv Yes for meaning, financial security, and healthy habits, I'm pricing in short timelines so not family

[Tao Lin @taoroalin]

@myhandle @nosilverv Wat, weirdos best normies in their early 20s all the time

[Tao Lin @taoroalin]

@aaronscher Why is it positive? I would think it's neutral / unclear sign because developing my talent is bad

[Tao Lin @taoroalin]

@TomDavidsonX Yeah, this is also my most salient disagreement with Plan A. Publishing all research will provide all actors with a better base of information than top labs have today, where they could find a bunch of low hanging research doesn't require large scale compute.

[Tao Lin @taoroalin]

@oitixion @nikolaj2030 @tejalpatwardhan IMO it did post-train Luna. The tools advance over time, today an instant pot + recipe from the internet still counts as "cooking dinner" even though it's far shallower work than what our previous generations did

[Tao Lin @taoroalin]

@MTSlive @QiaochuYuan Wat? Qiaochu is not an AI researcher?

[Tao Lin @taoroalin]

@nikolaj2030 @tejalpatwardhan Their screenshot is what I expected based on "sol trained luna". In a large org, there's a lot of division of labor, and many actions are just configuring your colleagues' code and make sure nothing breaks

[Tao Lin @taoroalin]

@nikolaj2030 My uncertainty or disagreement is about how much political will you need to shut down black sites in arbitrary countries, and what do we need to do to generate that much political will.

[Tao Lin @taoroalin]

@nikolaj2030 Eh that many humans want North Korea to not have nukes and they still have nukes, I think the political will needed is very strong. Conditioning in >=political will to fight Nazis across most countries, I agree we can stop them

[Tao Lin @taoroalin]

@_NathanCalvin well, there are rumors that they are less sparse than current top Chinese models

[Tao Lin @taoroalin]

@_NathanCalvin i don't want to comment on their sparsity or total tokens, ask fable/sol

[Tao Lin @taoroalin]

@nikolaj2030 Also, I think Plan S requires deep common knowledge that capabilities research is evil and societal self-policing. The main Plan S page doesn't mention that, and I am not hopeful about Stop AI held up by government surveillance instead

[Tao Lin @taoroalin]

@nikolaj2030 And I also see a lot of low hanging fruit in propensity training and scaffolding, and overall I expect a black site to be able to make a lot of progress with a few hundred people and a small amount of compute over 10 years

[Tao Lin @taoroalin]

@DKokotajlo I'm so glad you all got this published today!