midnight. teeth brushed, laptop closed, about to turn off the light.
ping.
"You've made a new sale ($15.83)!"
I sat on the edge of the bed and stared at it.
checked it wasn't a phishing thing.
checked the polar dashboard.
yeah — real.
someone, somewhere, just paid $19 for a thing
I built because I couldn't find one that worked.
didn't sleep well that night.
best kind of insomnia tbh.
22 days later: 9 orders, $77.28.
every ping since hits as hard as that first one.
if you're shipping something nobody's looking at yet:
the first ping is closer than you think.
https://t.co/hdC91fD804
boring origin story:
I had 200+ Claude conversations I wanted to back up.
Some with artifacts (docx, xlsx, SVGs).
Some with research PDFs I'd uploaded.
Tried to find a tool. CWS, GitHub, r/ClaudeAI, Indie Hackers.
Single-conversation → PDF? everywhere.
Batch export + bundle attachments into a ZIP? nowhere.
So I built it.
Apr 22, 5:41pm — init repo
Apr 22, 9:31pm — first paid feature shipped (attachment bundling)
Why was attachment bundling the first paid feature?
Because it was the exact thing I couldn't find a tool for that night.
If I'd pay for it, someone else would too.
📎 The two highlighted features in the screenshot —
that's literally what I was searching for and couldn't find.
Now I run my own extension to back up my Claude history every week.
Pretty satisfying tbh.
https://t.co/hdC91fD804
@istdrc It's a great product idea, I'd love to try on my project with AI agent. I'm just a bit concern about whether I could safely run the deamon process on my working machine...
Excited to release new repo: nanochat!
(it's among the most unhinged I've written).
Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.
It weighs ~8,000 lines of imo quite clean code to:
- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.
Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.
My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.
Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.
AI coding is used to generate a lot of bulk code that is often blindly accepted, but it seems there is at least as much opportunity for AI to help make codebases more beautiful.
Keeping a codebase in great shape or cleaning up a hairball takes a lot of effort beyond raw functionality, and a tireless AI assistant continuously pouring over everything looking for places to suggest changes should be valuable. AI as a diligent team member instead of your coding genie.
While there are factors peculiar to each’s understanding, I believe there are common coding behaviors that improve understanding for both humans and LLMs. It should be possible to run actual objective experiments with “style guides” for LLMs, then intersect them with the politics and fashion of human style guides.
Are there any tweaks to be made in LLM tokenization to more closely mirror programming language lexing?
Does saving context length settle tabs vs spaces?
I would like to see how a notoriously picky team like the @OpenBSD developers could onboard an AI team member.