Robert Gordan

test

[Robert Gordan @goodforzer0]

@tszzl yeah I’ve wondered the same. Like at an even more granular level, why did the models all converge on the same stylistic tics (eg em-dash). It’s counterintuitive to me that the path through the loss landscape would be so deterministic

[Robert Gordan @goodforzer0]

@DaveRBanerjee I mean filtering alone doesn't seem like a huge win if we're already in a data-constrained regime. The real breakthrough would be like synthesizing interesting RL problems for itself to solve.

[Robert Gordan @goodforzer0]

who knew the GPTs read christina rosetti

[roon @tszzl]

everyone is assuming this is some kind of quirk chungus marketing campaign but if you’ve worked with 5.4 and beyond they tend to call everything goblins, gremlins etc and it’s just super noticeable and if you work with them all day you start to get annoyed

[Robert Gordan @goodforzer0]

@leothecurious @sainingxie why can't you train a language model from nothing? after all, evolution did it

[Robert Gordan @goodforzer0]

@seconds_0 @willdepue @teortaxesTex I hope not.. The probability of winning is quite low anyways, participating and sharing interesting results along the way is probably a better way to get recognition.

[Robert Gordan @goodforzer0]

@varunneal @Sam_Acqua Honestly though by default I'm a bit skeptical that there's that much headroom from cross-document TTT, there just aren't that many val tokens.

[Robert Gordan @goodforzer0]

@varunneal @Sam_Acqua Yeah that's exactly the other bug I mentioned. Isn't clear to me whether the adapter could see the suffix. Author says it can't. https://t.co/4xx2Ma728U

[Robert Gordan @goodforzer0]

@molochofficial Obviously bad, but how would you describe what it’s doing here? To me: overwritten, stuffed with metaphors that don’t land. Every sentence seems to be similarly structured (independent clause, dependent clause xN)

[Robert Gordan @goodforzer0]

I'm sure others have said it too but weirdly enough I really love the personality of these coding models.. "I spent a long time trying to find a counterexample.." sounds like something my TA in school would say while giving me partial credit. This is Claude but 5.4 is v similar.

[Robert Gordan @goodforzer0]

@Butanium_ @voooooogel that’s such a good example of the unintuitive generalization ability that models can have. easy to take it for granted these days but I feel like it’s quite related to what was so magical about the first instruct tuned models

[Robert Gordan @goodforzer0]

now if only someone can pay for an h100 to see if this works

[Robert Gordan @goodforzer0]

Sorry bro not ambitious enough. I created auto-autoresearch-research, an agent that optimizes your agent that optimizes your research code. https://t.co/eKaTFS4lUW

[Andrej Karpathy @karpathy]

oh yeah i should have linked autoresearch probably
https://t.co/YCvOwwjOzF
(you don't "use it" directly, it's just a recipe/idea - give it to your agent and apply to what you care about.)

and the tweet about it that went mini-viral over the weekend with more context
https://t.co/q5eWsvx5p2

[Robert Gordan @goodforzer0]

not really

but I still wrote about it https://t.co/1BntiB2CaQ

[Robert Gordan @goodforzer0]

Can you replicate an ICML paper with less than $20 in runpod credits? I tried and the answer is

[Robert Gordan @goodforzer0]

obvious analogy to RLHF unlocking the latent intelligence from older pretraining-only models

[Robert Gordan @goodforzer0]

More and more I'm agreeing with alignment by default. The pet theory I have is that "alignment" will be more about eliciting the capabilities of next-generation models than about their ethics.

[Robert Gordan @goodforzer0]

@belindazli Hi Belinda, I think this is a really cool area. Do you think that self-supervised training to improve introspection could generalize to improvements in arbitrary domains?

[Robert Gordan @goodforzer0]

... We must expect great innovations to transform the
entire technique of the arts, thereby affecting artistic invention itself and perhaps even bringing about an amazing change in our very notion of art." -- Valéry, 1928

[Robert Gordan @goodforzer0]

"Our fine arts were developed, their types and uses were established, in times very different from the present, by men whose power of action upon things was
insignificant in comparison with ours..