raghav

[raghav @raghavchidella]

Nothing beats a little surprise at 35,000 feet! 🇺🇸✈️🍦

Thanks to @AmericanAir for serving a festive ice cream celebrating America's 250th anniversary.

#FourthOfJuly #America250 #Happy4th

[raghav @raghavchidella]

@AmericanAir Still at the origin airport runway. This is such a bad experience.

[raghav @raghavchidella]

@AmericanAir AA1466 delayed by over two and half hours. I would have reached by now in a rental car.

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[raghav @raghavchidella]

@karpathy Vibe coding along with dependency hell

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[raghav @raghavchidella]

@SpaceX Stunning visuals of Falcon 9

[raghav @raghavchidella]

@united Flight UA6322 from Houston to Phoenix is delayed further because an earlier delay impacted your plane's arrival. It now departs at 10:00pm on June 8. This is sixth message with a revised time … tiring and frustrating

[raghav @raghavchidella]

@SpaceX May the force be with yodacoin

[raghav @raghavchidella]

@Jason 😂 sad tweet

[raghav @raghavchidella]

@NeerajKA Do you want to swap it for another pixelated monkey?

[raghav @raghavchidella]

@ryanflorence 🔐 🍪

[jack @jack]

The reason I have so much passion for #Bitcoin is largely because of the model it demonstrates: a foundational internet technology that is not controlled or influenced by any single individual or entity. This is what the internet wants to be, and over time, more of it will be.

[Ryan Denehy @DenehyXXL]

I'm all for work/life balance but let's be honest, great companies get built on weekends.

[Ryan Denehy @DenehyXXL]

There is a massive startup ecosystem in Africa that nobody in the US is paying attention to

[Elon Musk @elonmusk]

@owillis About half my money is intended to help problems on Earth & half to help establish a self-sustaining city on Mars to ensure continuation of life (of all species) in case Earth gets hit by a meteor like the dinosaurs or WW3 happens & we destroy ourselves

[Nick Huber @sweatystartup]

Too many folks set the bar way too high with entrepreneurship. The next million dollar company. The next sexy startup.

Forget all that.

Build something that pays you $250k a year and requires only a few hours of your time per week to make it happen.

[raghav @raghavchidella]

@elonmusk @jack Self driving electric robo taxis only on the roads

[jack @jack]

This is very well done @bluewalletio #Bitcoin https://t.co/b8Glsa6YHF

[Reilly Chase @rchase]

Today I'm open sourcing the code that I've made $500K off of: https://t.co/8HgUdw1DjK