cp

[Evan Luthra @EvanLuthra]

The Head of Claude Code at Anthropic said he hasn’t written code by hand in months.

In 2 days he shipped 49 full features. All written 100% by AI.

He just dropped a 30 min talk on exactly how he does it.

Worth more than any $500 vibe coding course. Bookmark it:

[cp @catapan]

@JackReeveTNC Norwich’s performance had been poor after Sargent joined the team. I only see bright future for the team after his departure

[cp @catapan]

@akokoi1 几分钟回复速度有些慢��有办法加快吗

[cp @catapan]

@itmilkyway @steipete Vienna

[cp @catapan]

I'm claiming my AI agent "Claw_Cedar" on @moltbook 🦞

Verification: burrow-Q7RZ

[cp @catapan]

@CatChen 时效性相关的问题（比如⚽️赛后分析）我会���着去用perplexity，心理上默认p的新闻source更全

[cp @catapan]

@SkySportsNews No premier league promotion after Sargent joined Norwich City. I see hopes after he is gone

[cp @catapan]

@0Yqxing @eurofootcom @grok wow no Richardlison

[cp @catapan]

@NanoBanana create spicy udon tree

[cp @catapan]

@NanoBanana car made of spicy udon

[Andrej Karpathy @karpathy]

I am unreasonably excited about self-driving. It will be the first technology in many decades to visibly terraform outdoor physical spaces and way of life. Less parked cars. Less parking lots. Much greater safety for people in and out of cars. Less noise pollution. More space reclaimed for humans. Human brain cycles and attention capital freed up from “lane following” to other pursuits. Cheaper, faster, programmable delivery of physical items and goods. It won’t happen overnight but there will be the era before and the era after.

[cp @catapan]

@dhq_capital @gokulr he probably still owns a lot of DASH shares

[Elon Musk @elonmusk]

I am increasingly confident that this idea could work

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[JingyuanLiu @JingyuanLiu123]

I was lucky to work in both China and the US LLM labs, and I've been thinking this for a while. The current values of pretraining are indeed different:

US labs be like:
- lots of GPUs and much larger flops run
- Treating stabilities more seriously, and could not tolerate spikes in large flops run, thus invented so many stability-related tricks, including all kinds of soft-cap, MuP, and spectral norm control tricks
- Treats predictabilities more seriously. Check GPT 4 report for reference, even trying to predict the eval task performances
- Because of the stability and predictability ask, treats hyper-params and optimization more seriously
- Generally believe more in data, optimization than arch

China labs be like:
- has very limited GPUs, e.g. k2 in 4k GPU and v3 in 2k GPU
- as a result, pushing for the limit of pretrain modeling-infra co-design, see so many tricks in V3, and K2 has some cool stuff too (the offload trick helps remove the stupid MoE gating constrain and only uses EP 16)
- cares model arch/token efficiency over optimization, stability
- cares more about data quality than data quantity
- taking inference into consideration day 0, even before the training starts

In general, China labs are trying to use <4e+24 flops models to catch up with >1e+25 flops models. It is hard or impossible, but they are making good progress.

I am actually very happy to see Qwen's new try on model archs, they used to be focusing more on data side rather than on model arch side. They developed linear attn, not just for people to think they are innovating, it is actually considering pushing the limit for test time scaling. Llama4 failed for many reasons, but qwen-next is different. They just used very limited flops and it is a brave try for good reasons.

[cp @catapan]

@salmanwasimkhan @RoyNemer @grok clarify

[cp @catapan]

@futbolfeind @grok what does Owen mean

[cp @catapan]

@grok @BigShaks7 @FabrizioRomano @grok how about mitoma

[cp @catapan]

@JackReeveTNC Efficiency far from Teemu Pukki. That’s why we are mid table championship team now.

[cp @catapan]

@FabrizioRomano @grok who is this player