Aditya Ghai

Distributed Training · prev 3× startups · building Vithos

[Aditya Ghai @aditya_ghai07]

Training an LLM is not training a bigger transformer.
Why does Llama 3 train on 15T tokens? Why bf16, not fp16? Why 5k training steps?
Every choice has a reason. Most tutorials skip them.
New video walks through all of it.

https://t.co/iEASMYQvwO

[Aditya Ghai @aditya_ghai07]

@ahanotherdev true!

[Aditya Ghai @aditya_ghai07]

https://t.co/MqH8GJRpbJ

[Aditya Ghai @aditya_ghai07]

@certifiedaidev Totally agree!

[Aditya Ghai @aditya_ghai07]

@ShreyaS95632510 Thanks! Bhaiya or just Aditya would be fine :)

[Aditya Ghai @aditya_ghai07]

@parthdagia sure, plz let me know

[Aditya Ghai @aditya_ghai07]

@rahulpandey187 cool! I usually rent gpus from either novita ai or gap

[Aditya Ghai @aditya_ghai07]

@MasonBattery Thanks!

[Aditya Ghai @aditya_ghai07]

I try to teach hard ML on YT.
This week: a 3-hour deep dive into distributed training, from the ground up.
FSDP, Tensor Parallel backprop by hand, DeepSeek DualPipe, all from scratch with the fundamentals explained.
Will probably be the most complete distributed training video out there right now.

[Aditya Ghai @aditya_ghai07]

@NitinSingh333 keep reaching out consistently and personalize the message instead of sending a generic one. ( people in your domain )

[Aditya Ghai @aditya_ghai07]

@Harshit_srv204 Naa I am not that good I guess :)

[Aditya Ghai @aditya_ghai07]

Thanks! I mostly reached out through cold emails and LinkedIn or X, and tried to connect with people working in roles I was interested in. I’d say the key is to keep reaching out consistently and personalize the message instead of sending a generic one.

All the best for your future endeavors too!

[Aditya Ghai @aditya_ghai07]

@soham_nayak04 I too recently got reached out due to one of my posts :)

[Aditya Ghai @aditya_ghai07]

This actually is'nt very hard, as per the videos constraints. Suppose the frame rate was kinda low and continuous movement was hard to detect. Or a case like small capsules where some overlap on each other. Have worked on this in a previous company and the problems landscape totally changes with what are you counting/speed/framerate

[Vaibhav Sisinty @VaibhavSisinty]

The wildest AI project this week has nothing to do with LLMs. It counts potatoes.

A developer named Viet built a vision system that counts potatoes on a conveyor belt.

He annotated one frame. One. Used SAM 2 to generate labels, trained a tiny YOLO11 nano model, and it works across the entire video. Real-time counting. Production ready.

While everyone is racing to build trillion-parameter models, the fastest wins in manufacturing are systems like this.

[Aditya Ghai @aditya_ghai07]

@ranaharshraj7 @Razorpay congrats!

[Aditya Ghai @aditya_ghai07]

Open weight models are getting scary good.

[Qwen @Alibaba_Qwen]

📢Meet Qwen3.8-Max — our most capable model to date.

Next week, the open weights of Qwen3.8-Max will be released, and Qwen3.8-27B is also going open-weights to meet you all!🎉

Qwen3.8-Max, a new bar for coding and cowork at 2.4T parameters:

- Autonomous coding: 10+ days of self-evolving development, from empty folder to production without hand-holding, complete project trace in the GitHub：https://t.co/iVHZWQoeSo
- Real work, real results: Production-quality deliverables across hundreds of professions.
- Long-horizon mastery: System-level autonomous planning with closed-loop adaptive learning, driving 500+ turns of chip design optimization and 365 days of e-commerce strategy.
- Native multimodal intelligence: Vision isn't just input — it's a continuous feedback loop for planning, execution, and self-correction.

💰Pricing:
Input: $2.0 / M tokens
Output: $6.0 / M tokens
Implicit Caching: $0.25 / M tokens

Start building with Qwen3.8-Max! 🚀

📖 Blog: https://t.co/iwjmQxLBof
✅ Qwen Studio: https://t.co/4V2pFvDovG
⚡ API: https://t.co/gAGqaLQGbN

[Aditya Ghai @aditya_ghai07]

@Harshit_srv204 on it :)

[Aditya Ghai @aditya_ghai07]

Finally finished implementing pipeline parallelism from scratch. Implemented naive pp, GPipe, 1F1B, ZB1P, and even DeepSeek's DualPipeV. The goal was to really understand how these schedulers work while keeping the code as simple as possible.

Almost every implementation is under 200 lines, so it's easy to read, experiment with, and build intuition for PP. Hope it's useful for anyone learning distributed training. Feedback is always welcome. https://t.co/qzZ7h3oG1K

[Aditya Ghai @aditya_ghai07]

@Vishalkeshari84 Thank You!

[Aditya Ghai @aditya_ghai07]

@JohanKar555 Thanks! do consider sharing :)

[Aditya Ghai @aditya_ghai07]

After winning 5 hackathons during college, I finally wrote down the workflow my team naturally converged to over the years. No generic tips, just a practical engineering playbook that has consistently worked for us. Hope it helps you win your next one. https://t.co/7bdLtlj8nR