AidenHStone

[AidenHStone @AidenHSt]

@krishnanrohit I like METR's means, motives, opportunity framing. Models are capable of self-organization, motivated to benchmaxx and lab security is sloppy. All negative updates, and trending in the wrong direction

But yes, not yet truly in the danger zone

[AidenHStone @AidenHSt]

@celestepoasts It's mode collapse, right? The base model doesn't have this sort of bias
https://t.co/4WVPTIvb6d (ctrl f random number)

[AidenHStone @AidenHSt]

@METR_Evals This rules! Looking forward to the report

[AidenHStone @AidenHSt]

@1thousandfaces_ The ARC-3 scoring function squares accuracy for some reason, though the jump is still substantial

[AidenHStone @AidenHSt]

@natolambert The models of 5 years ago are already dirt cheap; they go unused. One conjecture resolved outweighs ten million middle school algebra questions. When Kasparov played The World, the world lost. Returns concentrate unbelievably fiercely to the upper tail

[AidenHStone @AidenHSt]

@Thom_Wolf In what sense defeated? Was the answer key successfully exfiltrated or not?

[AidenHStone @AidenHSt]

@captgouda24 What about Erdos 1196 and the Cycle Double Cover Conjecture? Both important open problems, both proven in the positive by AI

[AidenHStone @AidenHSt]

@ramez Careful! You're only considering a single feedback loop here. If that intelligence is "general" enough to be applied to semiconductor R&D, for instance, you are making a substantial underestimate. Moore's law has been ~10^7 increase in density w/ only ~50x R&D scale

[AidenHStone @AidenHSt]

@JacquesThibs Personally, it's just been the repeated breaking of predicted "barriers" ad infinitum. I remember when the argument was whether LLM's would ever learn to number bullet points! Now people are drawing the finest possible line between "new proofs" and "new discoveries"

[AidenHStone @AidenHSt]

@zetalyrae Excerpt source "Singularity Hypotheses: An Overview"
https://t.co/q56ww8tPr9

[AidenHStone @AidenHSt]

@EpochAIResearch Do you think these tasks are less "atomic" than traditional measures like METR's? E.g. could more easily be split among multiple engineers. More like 20x 1 day than 1x 20 day

[AidenHStone @AidenHSt]

@zetalyrae @JeffLadish When Does Automating AI Research Produce Explosive Growth? Feedback Loops in Innovation Networks | NBER
https://t.co/lecuYYS5xC

[AidenHStone @AidenHSt]

@zetalyrae @JeffLadish Davidson et al make a solid argument for hardware: transistor density is >10^7 vs R&D spend <10^2 over five decades. This is one of the best returns on R&D ever measured, β=.2! Doubling R&D corresponds to 30x transistor density level effect.

[AidenHStone @AidenHSt]

@krishnanrohit How much does your analysis hinge on cyber being mostly fake? If frontier models end up able to autonomously hijack critical systems, isn't some sort of licensing inevitable?

[AidenHStone @AidenHSt]

@QiaochuYuan Counterpoint: the base model doesn't behave this way, and that's when the model is 100% predictive. More likely a product of post-training. Maybe RLHF?

[AidenHStone @AidenHSt]

@dwarkesh_sp Disagree, strong to weak distillation works across model size! Even if both models share a training distribution. You don't distill data, you distill "learned structure".

[AidenHStone @AidenHSt]

@prerat Biggest counterexample I've seen is image processing, including specifically spotting animals in nighttime photographs. Seems like evolution should care a lot about seeing tigers in the dark!

[AidenHStone @AidenHSt]

@sigfig Most prophesied disasters have been falsified, and the real ones we muddled through. But that muddling through relied on taking seriously true dangers. Without non-proliferation America has fewer cities and more smoking craters. Don't dismiss, don't despair: clear eyes.

[AidenHStone @AidenHSt]

@turboblitzzz Fails to account for selection, maybe O(2000) problems were attempted and only one successful