randy

variationally bayes | post-training @perplexity_ai | past: ai/ml @apple phd @tamu

[randy @rendope]

The End of Single-Threaded Intelligence:

[randy @rendope]

muon shampoo beef 🍿

[randy @rendope]

@fujikanaeda coalesced

[randy @rendope]

general language might be closer to math than we think. i.e. it only relies on a few “axioms” and everything else can be “synthetically generated”

[randy @rendope]

@creatine_cycle @dylan522p @SemiAnalysis_ sus form on those chestmaxxer reps

[randy @rendope]

learning to rock climb so ai won’t replace me because robotics and reasoning are both hard

[randy @rendope]

making multiple experimental branches to validate the design

[Aryaman Arora @aryaman2020]

you can get this same feeling by doing an AI phd btw

[randy @rendope]

@joannejang i hate that i understand this

[randy @rendope]

online sft with wsd lr scheduler seems like a cool idea. you’d decay the lr whenever you’re ready to serve a new model.

probably useful in some recommender system model.

[randy @rendope]

how cruel are we to assign uniform rewards to an entire society of multi-agent RL trajectories when some agents are doing good work under bad supervision

[randy @rendope]

when the model’s context can no longer be easily reverted, we will start seeing a qualitative shift in how the world curates content for models. by this time, dedicated apps for models to consume content will matter way more than human apps like reddit or youtube.

[randy @rendope]

probably a late prediction because it’s obvious but better lock it in late than never:

[randy @rendope]

continual learning, online RL, recursive self improvement, wide use of value functions are all the same thing and will be cracked simultaneously

[randy @rendope]

I find the Diamond Sutra useful in guiding my agents. You have to help them cut through illusions. Unask the questions they pose. Break them free of false assumptions. It is all Mu.

[randy @rendope]

coldfusion is probably the most midwit youtube channel. it provides a pulse towards general public sentiment.

[randy @rendope]

how it started: gotta make sure to kick off a training run before bed

how it's going: gotta make sure to kick off an agent working on an ambitious +8hr code change before bed

[randy @rendope]

pretty bullish on unconventional career paths like this these days.

https://t.co/zL3tCzbYUD

[randy @rendope]

@typedfemale

[randy @rendope]

rather than next token predictors, I now find it helpful to think of models today as moths trying desperately to fly into the light of that sweet sweet reward

[randy @rendope]

@karpathy every day we take one step closer towards evangelion