Just open-sourced my weekend project: an ESP32 robot arm that expresses emotions using Gemini AI! 🤖🎭
Tell it "wave hello" or "show excitement" and watch it come alive with 20+ expressive gestures.
$40 robot arm + AI = endless fun
https://t.co/eTLrGpW9Se
#robotics#ESP32 #GeminiAI #OpenSource #MicroPython #DIY #maker
More on CNN history: https://t.co/HdCanIaCCl Why did "modern" CNNs emerge in Japan 1979-1988 (Fukushima's 1979 CNN architecture → Zhang et al.'s 1988 backpropagation-trained CNN)? Let's look back at the 1980s. Back then, Japan was the envy of the world. Before the 1990 crash, the Tokyo stock market was the world's largest, and the world's 6 most valuable public companies were all Japanese. So were the world's richest business men. According to the real estate market, tiny Japan was 4 times more valuable than the much bigger US. The central square mile of Tokyo had the value of California. Japan had far more robots than any other country, and by far the most expensive AI project: the 5th Generation Project. Interestingly, this project had little to do with neural networks; it was mostly about logic programming and expert systems. So Fukushima and colleagues were outsiders back then. But at least there was sufficient funding in Japan, even for such unpopular types of blue skies research. Today, the rest of the world can be thankful for that. See also https://t.co/3Xfc5qoWQT
💀🤖 Meet Kenny, my co-worker in the longevity space. After countless late nights wrestling with AI pipelines, integration headaches, and “how do we make this explainable and reliable for longevity practitioners?”… Kenny didn’t quite survive.
Building in longevity isn’t just about biology, it’s about data:
Training AI models that can process complex biomarker sets.
Making sense of diagnostics in real-time for practitioners.
Every challenge feels like another piece of skeleton work, pulling frameworks apart and stitching them back together. Sometimes, it really does feel like you could end up like Kenny.
But here’s the good part: we’re thriving, learning, and solving problems that will actually help people live healthier, longer lives.
👉 Tell me: what’s the toughest AI challenge you’ve faced that nearly turned you into Kenny?
Activate to view larger image,
If this Karpathy interview doesn't pop the ai bubble,
nothing will.
10 brutal quotes:
1. LLMs don’t work yet
They don’t have enough intelligence, they’re not multimodal enough, they can’t use computers, and they don’t remember what you tell them.
They’re cognitively lacking. It’ll take about a decade to work through all of that.
2. When you boot them up, they always start from zero
They have no distillation phase, no process like sleep where what happened gets analyzed and written back into the weights.
3. What’s stored in their weights is only a hazy recollection of the internet
It's just a compressed blur of 15 trillion tokens squeezed into a few billion parameters. Their context window is just short-term working memory.
4. They’re good at imitation, terrible at going off the data manifold
Too much memory, not enough reasoning.
We need to strip away the memorized knowledge and keep the cognitive core: the algorithms, the magic of intelligence, problem-solving, strategy.
5. We’ve probably recreated a cortical tissue, pattern-learning and general, but we’re still missing the rest of the brain
No hippocampus for memory.
No amygdala for instincts.
No emotions or motivations.
6. They memorize perfectly but generalize poorly
If you give them random numbers, they can recite them back. No human can do that.
That’s the problem: humans forget just enough to be forced to find patterns.
7. Anything truly new, code that’s never been written before, ideas that have no template; they stumble
They’re still autocomplete engines with perfect recall and no understanding. Until we find that cognitive core, intelligence stripped of memory but full of reasoning, they’ll stay brilliant mimics, not minds.
Respect for your honesty. 🙏 Too many people only show the wins and hide the pain. That’s why we built https://t.co/CpTYUmzPFc a free tool to track which finfluencers are real vs. who’s just selling you dreams. You can see their calls, win rates, and credibility before trusting anyone again.
No more glorifying fake screenshots or cherry-picked gains.
Excited to release new repo: nanochat!
(it's among the most unhinged I've written).
Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.
It weighs ~8,000 lines of imo quite clean code to:
- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.
Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.
My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.
Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.
And before anyone complains, "which another AI rates" is a bit of an oversimplification (character limits!): as explained in the diagram, the free text response from the AI acting like a consumer is converted into embeddings & compared to reference statements with Likert ratings