Vishvanand

information retrieval systems and indexing systems at a company you’ve heard of. train models in my free time. opinions are my own

[Vishvanand @Vishvanand]

or they’re just being truly benevolent, the internet being free and open did provide benefits to everyone, why not LLMs?

[Vishvanand @Vishvanand]

why would china want to open source this extremely powerful model instead of running a duopolistic geopolitical race?

think about which country benefitted the most by concentrating global intelligence in one place and distributing it globally via the internet and what happens when that surface area grows too large and super intelligence becomes ambient

[Vishvanand @Vishvanand]

we don’t think hard enough about what it means for agents to make more purchasing decisions and capital allocation decisions than humans

[Vishvanand @Vishvanand]

did china just wink at NVIDIA’s supposedly unassailable CUDA moat with this section in the kimi blogpost?

[Vishvanand @Vishvanand]

@Sauers_ they get rid of it if you do anything that can be construed as ML research or hit classifiers is my theory

[Vishvanand @Vishvanand]

it’s kinda depressing how fable in its infinite wisdom thinks the best startup to create in the AI era to combat the bitter lesson is…

another RL training env company.. gg

[Vishvanand @Vishvanand]

@ben_rapaport true, verifiable in that sense. the harder condition is RLing it effectively.. code tests parallelize and pass/fail the same whenever you run them, stock picks don’t, and you don’t get many time steps to learn the current regime

[Vishvanand @Vishvanand]

if you believe in the arguments that coding fell to large labs because they focused on it, it should make your jaw drop that jane street is building a massive gpu cluster

me personally, i still cope that coding fell because of verifiable rewards and not because of the allure of RSI

[Vishvanand @Vishvanand]

@ben_rapaport and that labs focused on coding because they believe it will lead them to RSI

[Vishvanand @Vishvanand]

im suggesting that if the only thing stopping LLMs from getting better at generating stock market alpha is directed focus and compute, jane street would make a ridiculous amount of money

but coding might be a substantially easier domain for LLMs to crack because they have verifiable rewards and can be RLed effectively

[Vishvanand @Vishvanand]

it’s a really good model!! congrats thinky and 🇺🇸

[Vishvanand @Vishvanand]

it’s fun watching x debate in real time about why they’d release this model if it’s not good as good at coding as GLM..

if you’ve been paying attention to LLMs act 3 after coding (or if you think about astrology like me), it becomes entirely obvious

[Thinking Machines @thinkymachines]

Today, we are introducing Inkling.

Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.

https://t.co/Ghebq5mG30

Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

[Vishvanand @Vishvanand]

and as expected, passes my astrology test with flying colors

[Vishvanand @Vishvanand]

it’s awesome that you can train retrievers using alphago style flywheels or as fable calls it “ratchet” using a single A100-80G

[Vishvanand @Vishvanand]

it’s kinda fun that GPUs have gotten so good that you can train really good retrievers

I’ve experimented with contrastive embedding models, asymmetric encoder heads, generative retrieval and my current favorite is alpha-go style training of retrievers after i read the harness-1 paper (until then i avoided RL like the plague)

then i watched eric jang on dwarkesh’s podcast and read his blog and this slide made me alphago pilled

[Vishvanand @Vishvanand]

i took a stanford class and ranked second in a class of 200 students back in the day in a RAG task (yuck I know) by just calling gpt-4o API and adding some scary language to the prompt

others trained models, complex systems, multihop, graph RAG and yelling at the LLM worked best

in retrospect, i never internalized the bitter lesson until now when i routinely pull out pencil and paper to understand what LLMs teach me