Divya Makkar

@LumaLabsAI | cs @uwaterloo, prev reasoning @cohere

[Divya Makkar @_DivyaMakkar]

@baaadas @LumaLabsAI It was great working with you! Excited to see what's next!

[Divya Makkar @_DivyaMakkar]

@_pavidhiman Woah, this is so cool!

[Divya Makkar @_DivyaMakkar]

@_rajanagarwal @ProximalHQ very cool!

[Divya Makkar @_DivyaMakkar]

@_pavidhiman so cool!

[Divya Makkar @_DivyaMakkar]

@AahaanMaini @socraticainfo 🔥🔥🔥

[Divya Makkar @_DivyaMakkar]

@adiprasadd @modal @hackwithtrees @Waymo @TencentGlobal @kyanyang_ @kevinjosethomas @shreybirmiwal congrats!

[Divya Makkar @_DivyaMakkar]

@aarush @Meta Congrats Aarush!

[Divya Makkar @_DivyaMakkar]

@ElijahKurien Congrats Elijah!

[Divya Makkar @_DivyaMakkar]

@chinmayjindal_ @GroqInc @nvidia cfbr

[Divya Makkar @_DivyaMakkar]

@emm4x3 @xai @angelafeliciawu @stevenzhang Amazing!! Congrats!

[Divya Makkar @_DivyaMakkar]

@adiprasadd 🔥🔥

[Divya Makkar @_DivyaMakkar]

@LeoMq06 @rox_ai @windsurf @cohere So fire Leo

[Divya Makkar @_DivyaMakkar]

I worked on this with @AdityaMakkar000 and @chinmayjindal_. Take a look at our blog to see how you can go from zero-to-one scaling a transformer from scratch.

GitHub: https://t.co/pngCWGjAj0
Blog: https://t.co/N2hmijqwXp

[Divya Makkar @_DivyaMakkar]

I spent the past few months building JAXformer: One of the first open source guides on how to scale modern transformers in JAX.

Trained entirely on TPUs, it supports distributed ML, Ray tokenization, MoE, n-D parallelism and end-to-end inference.

Here’s how to do it:

[Divya Makkar @_DivyaMakkar]

We trained until we hit a 3.28 val loss (inspired by nanoGPT speedrun) over 26,100 steps with 6.5 billion tokens.

[Divya Makkar @_DivyaMakkar]

@_pavidhiman @PragmaClin @vanar_robots 🔥🔥🔥