Asterios

I enjoy learning new things and occasionally write about them.
I also write code. Sometimes it works.

[Asterios @asterias__]

I still can't believe I rely on this to make life choices

[Asterios @asterias__]

Health, love and money
https://t.co/a71X2dkKsS

[Asterios @asterias__]

@PalmyrPar In 18 years, we will have doubled the human life expectancy, so...

[Asterios @asterias__]

@conductr_ By the way, this is how I think the world could "look" since colors, sounds, and everything else are just interpretations of the brain

[kanav @kanavtwt]

the meme has been updated

[Asterios @asterias__]

If alcohol were discovered today, it would almost certainly be regulated as a psychoactive drug, not a food.

Under modern standards it would be classified as a carcinogenic CNS depressant with no risk-free level of chronic exposure.

While it would likely pass Phase 1 acute safety testing, it would never qualify for food-additive or GRAS status and would carry strong cancer and dependency warnings, similar to or stricter than tobacco.

[Asterios @asterias__]

What Actually Happens When You Drink Alcohol
https://t.co/9vuJKeRPG4

[Asterios @asterias__]

Happy world introver day!

#IntrovertDay #introvert

[Asterios @asterias__]

Happy World Introvert Day!

[Asterios @asterias__]

Happy New Year, my fellow introverts.
https://t.co/asFKIQdw2b

[Asterios @asterias__]

Happy New Year, my fellow introverts.
https://t.co/asFKIQdw2b

[Asterios @asterias__]

What Actually Happens When You Drink Alcohol
https://t.co/9vuJKeRPG4

[Asterios @asterias__]

@bryan_johnson I have a different rule: I don't eat sugar all year, and I only eat some during the Christmas holidays 🎄🍪☃️🎅

[Asterios @asterias__]

@WatcherGuru

[Asterios @asterias__]

Unless you're sweating heavily for over an hour, sick with fluid loss, or entering a sauna, you don't need electrolytes in your water. Regular meals provide enough sodium, potassium, and magnesium. Adding electrolyte mixes when you don't need them can lead to excess sodium. For most people, plain water is all that's needed for proper hydration.

[Asterios @asterias__]

@Rainmaker1973 It's incredible how evolution managed to figure this out

[Asterios @asterias__]

@waitbutwhy I'm still freaking out that color only exists in our brains and that the universe is actually colorless.

[Asterios @asterias__]

#veo31 this is supposed to be Moonlight Sonata. I didn't expect it to work, but imagine the day it does 🤯

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Asterios @asterias__]

@curiosityonx Ilúvatar

[Asterios @asterias__]

If the first screen in your app is a login screen, I'm deleting the app.

[Asterios @asterias__]

@haider1 One breakthrough in neuroscience and none of that will be true.