Ali Elnagmi

19 | incoming @AWS | cs @UWaterloo

[Aadi Kulshrestha @MankyDankyBanky]

I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more.

Wrote the full transformer architecture, and BPE tokenizer from scratch.

The framework features:
- Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x increased throughput
- Automatic WebGPU fallback for non-NVIDIA devices
- TypeScript API with Rust compute backend
- One npm install to get started, prebuilt binaries for every platform

Try out the model for yourself: https://t.co/TB2itlmCVT

Built with @_reesechong. Check out the repos and blog if you want to learn more.

Shoutout to @modal for the compute credits allowing me to train on 2 A100 GPUs without going broke

cc @sundeep @GavinSherry

[Reese Chong @_reesechong]

I implemented a 2D neural network from scratch with no ML libraries, just scalars + sgd in TypeScript

It mostly did not work (until I fixed it)

Here are the bugs I hit and what they taught me about how neural nets actually train

Thread🧵

[Ali Elnagmi @elnagmiali]

@_reesechong this is awesome, incredible visuals

[Ali Elnagmi @elnagmiali]

@hamzah_2x rebirth

[Ali Elnagmi @elnagmiali]

@suneruperera @TouchlineX

[Ali Elnagmi @elnagmiali]

@dundeez I want to start yoyoing

[Simon @Simon_Hypixel]

We did it. Hytale is saved. We have acquired Hytale from Riot Games.

[Ali Elnagmi @elnagmiali]

@armanishmae1 not adding apex to the list after last nights box???

[Ali Elnagmi @elnagmiali]

@x1angNFT bro my cases are to the moon rn

[Ali Elnagmi @elnagmiali]

this season is going to be crazy

[Ali Elnagmi @elnagmiali]

When I sleep 8h it feels like 4.
When I sleep 4h it feels like 8.

[Ali Elnagmi @elnagmiali]

this is interesting

https://t.co/lQJiQghRn7

[Rhys @RhysSullivan]

incredible how pretty much the entire github homepage is useless

[Ali Elnagmi @elnagmiali]

Throwback to 6 years ago when I used to make minecraft plugins and discord bots on 4gb. Now I'm struggling with 16...

[Ali Elnagmi @elnagmiali]

the grass is so green outside rn

[Ali Elnagmi @elnagmiali]

@jefcodes chairs are for the weak, youre ngmi bro