Koushik

[Sanjeev Sanyal @sanjeevsanyal]

Heading home after 3 amazing days at IIT Madras. The place is the most interesting intellectual cluster in India right now…. Just amazing things are happing there. The brain mapping project just blew my mind. Then there were Agnikul’s space rockets, ePlane’s drone taxis, and the first attempt to build an indigenous marine engine. Well done Prof Kamakoti @iitmadras - you now need to tell us the recipe so it can be replicated in other parts of the country

[Vijay Upadhyaya @VijayUpadhyaya]

@x_rahulraj Is it nature or nurturer, this family should end this debate !

[sunny horse @haavlihudga]

bangers from start to finish.

[Koushik @kpbhat25]

Wandering thoughts:

https://t.co/qGbzwVzVdR

[Koushik @kpbhat25]

@BMTC_BENGALURU The idea of one QR for one ticket isn't working. Takes too long for single user and bus halts to issue ticket for everyone.
Better time to revert back to the old scanner model.

[Giorgia Meloni @GiorgiaMeloni]

Thank you for the gift

[Koushik @kpbhat25]

Vijay uses Gemini or CHAT GPT to take advise instead of people so that he could form a government.

[Koushik @kpbhat25]

Vijay took part in the TN elections and the whole politics turned into a movie. Nolan who?

[Jordanreviewsittt @jordanreviewsit]

Commuting to the office to take virtual calls 😍

[shirish @shiri_shh]

Russ Hanneman wasn’t WRONG.

"It's not about how much you earn. it's about how much you're worth."

[Koushik @kpbhat25]

ಮಗಾ life alli background music ಜೊತೆ ಓಡಾಡಬೇಕು, ಇಲ್ಲಾಂದ್ರೆ background noise ಆಗ್ಬಿಡ್ತೀವಿ.

ಪ್ರದೀಪ್ Eshwar voice alli ಓದಿ.

[Koushik @kpbhat25]

Claude ಬಲ್ಲವನಿಗೆ ಅಪಾಯವಿಲ್ಲ

Codex ಬಲ್ಲವನಿಗೆ ತೊಂದರೆಯಿಲ್ಲ

ಆದರೇ...

Tokens ಇದ್ದಷ್ಟೇ ಕಾಲು ಚಾಚು!

[Koushik @kpbhat25]

Thakur from Sholay would be smiling

[Let's X OTT GLOBAL @LetsXOtt]

Dhanush's Expensive Hobby , WORTH IS 50- 60 Cr

During a movie promotion interview, a reporter noticed his watch and mentioned that it was worth around ₹2.5 crores. This led to the discovery that Dhanush has a huge collection of very expensive watches, which is estimated to be worth around ₹50 to ₹60 crores

"The very first watch I truly fell in love with was one my mother bought me when I was in school. It was a simple, plastic digital watch that cost less than a hundred rupees.
It would often run out of battery, but I loved that watch so much that I would still wear it to school even when it wasn't working anymore. Today, even though I collect many expensive watches, I still have that special first watch safely with me." — @dhanushkraja

[Koushik @kpbhat25]

My next vote will be to the government which starts Jira ticketing for tasks in the constituency.

[Koushik @kpbhat25]

Saw a guy smoking in an EV car, does that count as pollution?

[Koushik @kpbhat25]

Congress and BJP IT cell fight looks like two lovers status war on WhatsApp.
waiting for them to take it offline

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Koushik @kpbhat25]

It's so ironic to see every company has a sustainable practices team to make lesser carbon emissions,
But all the executives and leadership drive car to office.

[Koushik @kpbhat25]

DK Shi posting on Bengaluru,
And getting tweets on how bad administration is

[Koushik @kpbhat25]

Miyan so accurate!!