Michal Horecki

C++ CUDA autograd (GradCraft). 90M GPT on a 3090, ~45% eager PyTorch fp32.
18yo, CS freshman @ PW
Looking for remote part-time ML systems.

[Michal Horecki @h00recki]

i find that really cool <3 https://t.co/Eyx2vFNJCU

[Michal Horecki @h00recki]

i was soldering for the first time and it was such a botch job lmao

[Michal Horecki @h00recki]

@elliotarledge complete the cycle and make an AI that then extracts highlights from the gameplay and makes ig reels out of it, and another AI that watches the reels

[Michal Horecki @h00recki]

repo:
https://t.co/BGPnzL9WxB

[Michal Horecki @h00recki]

my tiled attention got gapped so bad

[Michal Horecki @h00recki]

On that flash attention type shi

[Michal Horecki @h00recki]

@benbye i just finished training my first clanker in an engine like this that i wrote

cool project

[Michal Horecki @h00recki]

GradCraft officially trained the first clanker

it is a prime slop generating machine

its a screen from README.md in my repo

finally proved gradcraft <3 finito

[Michal Horecki @h00recki]

do u benchmark the kernels you write against production grade ones? Im writing flashattn rn, and i find it funny that the idea is pretty much the same, but if I compare my code to pytorch i am still significantly slower because of 100 diff intrinsics, tensor cores, math tricks/insights, etc. that pytorch uses to make it fast

[Michal Horecki @h00recki]

I just wrapped up README.md for GradCraft

everyone on windows can now copy the code and train a clanker on their machine

is it better than pytorch? not even close. Is it my own? hell yea

link in bio

[Michal Horecki @h00recki]

i wrote naive gemm vs cublas and got smoked by a factor of 10x

now im getting into the tiled version, will see

[Michal Horecki @h00recki]

@RedJOe_0608 thats fire, are you going to write the tiled version too? im just getting into it

[Michal Horecki @h00recki]

OH MY FUCKING GOD THE CLANKER I PRETRAINED IN GRADCRAFT CAN WRITE PYTHON!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

it is a bit retarded on the binary search because it inverted update of left adn right

BUT OH MY GOD IT IS VALID CODE

IT EVEN WORKS WITH DIFFERENT VARIABLE NAMES

[Michal Horecki @h00recki]

@trashh_dev i just saw a tweet talking about slop-grenades and im laughing so hard looking at this

[Michal Horecki @h00recki]

I wrote an engine in C++ to train a GPT style LLM on my 3090, it was like 180M params, and today I decided to check inference on it, its like 40% training done
Its so stupid lmao, I learned to not just scrape github for small LLMs

PROMPT (pretrained so couldn't just ask it):
#include <string>
#include <algorithm>
// Optimal solution to reverse a string
std::string reverse_string(std::string s) {

Generating...
--- PROMPT---
#include <string>
#include <algorithm>
// Optimal solution to reverse a string
std::string reverse_string(std::string s) {
--- MODEL OUTPUT ---
nameInstr keyflagditionally { write_UnicodeString(keychar_cStrSQpack); false;}});
total_depth += step;

}
strides mu(min_batch, boundary_length* extent);
------

Later I wanted to print a buffer and got blasted with an unfinished game engine from 2018

[Michal Horecki @h00recki]

@flatkey101 the amount of optimizations you can add to an autograd engine is absolutely diabolical, my v1 of the gpt training script was like 60x slower than it is now

[Michal Horecki @h00recki]

Why a 90M GPT step went from 391ms to 177ms after I stopped mallocing every tensor. https://t.co/EuHErcwC45

[Michal Horecki @h00recki]

calling cudaMalloc in a training loop vs using a memory pool gave a 2x difference. Article below <3
177ms/step vs 361ms/step

core is: with cudaMalloc the CPU first waits for the GPU to finish going math, and then the GPU waits for the CPU to finish allocating memory and queueing up the kernel.

https://t.co/sTpTWbjmdM

[Michal Horecki @h00recki]

Why a 90M GPT step went from 391ms to 177ms after I stopped mallocing every tensor. https://t.co/EuHErcwC45

[Michal Horecki @h00recki]

the 180M model came out absolutely retarded after 40% of training, so i started training on python instead of cosmopedia and C++. I think it will be easier for it to grasp and won't have to memorize 67 676 676 biological names from cosmopedia

Also it trains 4x faster. 90M params

[Michal Horecki @h00recki]

slowly getting somewhere &lt;3