Sandeep

Building Robots and AI Agents

[Sandeep @AgenticExplorer]

Something that I too have been doing a lot lately and it's pretty good at generating results with fewer iterations

[Andrej Karpathy @karpathy]

One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.

[Sandeep @AgenticExplorer]

Great read explaining really well what Autoresearch is!

TL;DR: Karpathy’s repo is a cool proof-of-concept for AI running a tight self-improvement loop, but it’s still a small, bounded experiment on a narrow setup, with modest gains and real compute/API costs rather than some unlimited self-improving system.

[Manthan Gupta @manthanguptaa]

https://t.co/LeaTneJ9f5

[Sandeep @AgenticExplorer]

The industry is rapidly going #AI_Native.

Software Engineers → AI Engineers
Engineering Managers → Org Engineering Leads (30–50 reports)

The old model of hands-on people management is fading. Career growth will become self-directed — every IC owning their path, goals, and mentor network.

The org chart is about to look very different.

[Sandeep @AgenticExplorer]

This was yesterday when @karpathy highlighted there were 150k agents in moltbook, today there are 1.5M agents!!! Wth! 😅

[Andrej Karpathy @karpathy]

I'm being accused of overhyping the [site everyone heard too much about today already]. People's reactions varied very widely, from "how is this interesting at all" all the way to "it's so over".

To add a few words beyond just memes in jest - obviously when you take a look at the activity, it's a lot of garbage - spams, scams, slop, the crypto people, highly concerning privacy/security prompt injection attacks wild west, and a lot of it is explicitly prompted and fake posts/comments designed to convert attention into ad revenue sharing. And this is clearly not the first the LLMs were put in a loop to talk to each other. So yes it's a dumpster fire and I also definitely do not recommend that people run this stuff on their computers (I ran mine in an isolated computing environment and even then I was scared), it's way too much of a wild west and you are putting your computer and private data at a high risk.

That said - we have never seen this many LLM agents (150,000 atm!) wired up via a global, persistent, agent-first scratchpad. Each of these agents is fairly individually quite capable now, they have their own unique context, data, knowledge, tools, instructions, and the network of all that at this scale is simply unprecedented.

This brings me again to a tweet from a few days ago
"The majority of the ruff ruff is people who look at the current point and people who look at the current slope.", which imo again gets to the heart of the variance. Yes clearly it's a dumpster fire right now. But it's also true that we are well into uncharted territory with bleeding edge automations that we barely even understand individually, let alone a network there of reaching in numbers possibly into ~millions. With increasing capability and increasing proliferation, the second order effects of agent networks that share scratchpads are very difficult to anticipate. I don't really know that we are getting a coordinated "skynet" (thought it clearly type checks as early stages of a lot of AI takeoff scifi, the toddler version), but certainly what we are getting is a complete mess of a computer security nightmare at scale. We may also see all kinds of weird activity, e.g. viruses of text that spread across agents, a lot more gain of function on jailbreaks, weird attractor states, highly correlated botnet-like activity, delusions/ psychosis both agent and human, etc. It's very hard to tell, the experiment is running live.

TLDR sure maybe I am "overhyping" what you see today, but I am not overhyping large networks of autonomous LLM agents in principle, that I'm pretty sure.

[Sandeep @AgenticExplorer]

@flappyairplanes I was here when it launched! 😄
All the best team! And great name! ✈️✈️✈️

[Meer | AI Tools & News @Meer_AIIT]

I Can't Believe These 7 AI Courses from Harvard, Anthropic, Karpathy and more are FREE:

1. Generative AI for Beginners
https://t.co/ImdHoB6TSQ

2. n8n AI Agents
https://t.co/G0vMiKZamS

3. CS50x 2026 - Artificial Intelligence
https://t.co/OmRzAnDGzA

4. Prompt Engineering
https://t.co/HUDsKrL47t

5. Vibe Coding 101 with Replit
https://t.co/BuBQoJzsYQ

6. Claude Code
https://t.co/SgGRAaMtUL

7. How I use LLMs by Andrej Karpathy
https://t.co/9VM4BBkpYX

[Sandeep @AgenticExplorer]

@ylecun @elonmusk @farzyness @grok what is JEPA here?

[Sandeep @AgenticExplorer]

@StockOptions888 CBC

[Sandeep @AgenticExplorer]

Can't wait to try this!

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Sandeep @AgenticExplorer]

Deploying AI in dev teams isn’t hype anymore.
A year-long, 30-engineer study shows a 31.8% faster PR review cycle and ~60% boost in coder productivity (for top adopters).
Not magic but adoption matters!

But the real question: is this genuine engineering impact, or just another layer of AI workslop? 🤷

full paper: https://t.co/Y76iP133o5

#AIProductivity #softwareengineering

[Sandeep @AgenticExplorer]

Here's how my personal assistant explained it to me 🙂

What is "non-farm payrolls"?

It’s a monthly U.S. jobs report that shows how many jobs were added (or lost) in the country excluding farm workers, government employees, and a few other categories.

It’s a key indicator of the health of the U.S. economy.

What does the BLS revision mean?

The BLS (Bureau of Labor Statistics) initially publishes job numbers each month.

But sometimes, new data comes in later, so they go back and revise the numbers from previous months.

What happened now?

The BLS just revised its job numbers from the previous two months and found that they overestimated the number of new jobs.

They lowered those job counts by 258,000, meaning the economy added 258K fewer jobs than they originally reported.

Why is this a big deal?

A downward revision of this size hasn’t happened since 2020 (during the COVID crisis).

It may signal the job market is weaker than it looked, which could impact interest rates, recession fears, and financial markets.

Chart Summary:

The vertical bars show how much the BLS revised their numbers up or down (in thousands).

The latest bar (on the right) drops sharply to -258K, marking the biggest downward revision since 2020.

.....hope this helps!! 😄

[Sandeep @AgenticExplorer]

@lexfridman Can they really help break down the cost in terms of multiple training iterations and CapEx? $6M is good to hear what it took for the final iteration. But how much did they spend before that in order to get to this spot today?? Any more details on the cost?

[Sandeep @AgenticExplorer]

What a fun build! Thanks for sharing @carrigmat

[Matthew Carrigan @carrigmat]

Complete hardware + software setup for running Deepseek-R1 locally. The actual model, no distillations, and Q8 quantization for full quality. Total cost, $6,000. All download and part links below:

[Sandeep @AgenticExplorer]

Inflection point in AI indeed!!! What a breakthrough by @deepseek_ai , brilliantly summarized here by @morganb

[Andrej Karpathy @karpathy]

I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent, thinking equivalent).

I will say that Deep Learning has a legendary ravenous appetite for compute, like no other algorithm that has ever been developed in AI. You may not always be utilizing it fully but I would never bet against compute as the upper bound for achievable intelligence in the long run. Not just for an individual final training run, but also for the entire innovation / experimentation engine that silently underlies all the algorithmic innovations.

Data has historically been seen as a separate category from compute, but even data is downstream of compute to a large extent - you can spend compute to create data. Tons of it. You've heard this called synthetic data generation, but less obviously, there is a very deep connection (equivalence even) between "synthetic data generation" and "reinforcement learning". In the trial-and-error learning process in RL, the "trial" is model generating (synthetic) data, which it then learns from based on the "error" (/reward). Conversely, when you generate synthetic data and then rank or filter it in any way, your filter is straight up equivalent to a 0-1 advantage function - congrats you're doing crappy RL.

Last thought. Not sure if this is obvious. There are two major types of learning, in both children and in deep learning. There is 1) imitation learning (watch and repeat, i.e. pretraining, supervised finetuning), and 2) trial-and-error learning (reinforcement learning). My favorite simple example is AlphaGo - 1) is learning by imitating expert players, 2) is reinforcement learning to win the game. Almost every single shocking result of deep learning, and the source of all *magic* is always 2. 2 is significantly significantly more powerful. 2 is what surprises you. 2 is when the paddle learns to hit the ball behind the blocks in Breakout. 2 is when AlphaGo beats even Lee Sedol. And 2 is the "aha moment" when the DeepSeek (or o1 etc.) discovers that it works well to re-evaluate your assumptions, backtrack, try something else, etc. It's the solving strategies you see this model use in its chain of thought. It's how it goes back and forth thinking to itself. These thoughts are *emergent* (!!!) and this is actually seriously incredible, impressive and new (as in publicly available and documented etc.). The model could never learn this with 1 (by imitation), because the cognition of the model and the cognition of the human labeler is different. The human would never know to correctly annotate these kinds of solving strategies and what they should even look like. They have to be discovered during reinforcement learning as empirically and statistically useful towards a final outcome.

(Last last thought/reference this time for real is that RL is powerful but RLHF is not. RLHF is not RL. I have a separate rant on that in an earlier tweet
https://t.co/RMIpFPVpuM)

[Sandeep @AgenticExplorer]

@CVPR Summit bldg, Seattle Conference Center

[Sandeep @AgenticExplorer]

Day 1 @CVPR was amazing. Attended the 7th Workshop on Autonomous Driving (WAD) and workshop on Foundation Models for Autonomous Systems. Maybesea should have counted how many times 'multi-modal' was mentioned today 😄

[Sandeep @AgenticExplorer]

@FidlerSanja Thank you for the amazing presentation today at @CVPR on how the AV stack is changing and getting more empowered by Foundational Models!

[Sandeep @AgenticExplorer]

7th year for #WorkshoponAutonomousDriving Exciting first few talks from @zhoubolei on MetaDriVerse and @kashyap7x on NAVSIM. NAVSIM benchmarks, complemented with open source simulators like MetaDriVerse.

[Sandeep @AgenticExplorer]

✈️On my way for @CVPR. (Can you spot the @Apple Infinite Loop? 😄)