this idea, but wanted the same for LLM training 👀
so I built Tensorforge.
design a model, wire up GPUs, change precision/batching/parallelism. then watch memory, throughput, time, and $$$ update live.
an LLM training playground with no real GPUs required.
demo link below
everyone's drawn a URL shortener on a whiteboard. ever watched one DIE though
built mine in breakscale (my system design simulator where the diagram actually RUNS). all good at 50 rps... crank it to 500 and the p99 just explodes lmao
then cache + load balancer + replicas and suddenly it's eating 5000 rps like nothing. every number on screen is real, nothing is faked
free, runs in your browser, go break something: https://t.co/tCGrCpe0W8
Interesting direction for LLM inference: Dory 1.0 is a 4B model that gets more compute by looping through shared layers instead of generating longer reasoning traces.
same weights, variable depth, better performance. worth a read 👀
Link: https://t.co/GbOuAISoWh
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
day 57 of AI eng - following generation
finishing this section with the inference loop for an encoder–decoder transformer.
encode the source once. start the decoder with a start token, predict the next token from the current prefix & encoder memory, append it, repeat.
the decoder produces vectors for the prefix, but we use the final position's vocabulary distribution to choose the next token. stop at an end token or a length limit.
training can use known shifted targets. inference feeds back predictions, so the prefix grows on every step.
from a forward pass to a complete sequence, one token at a time.
day 56 - the full transformer decoder
three sublayers in order:
1. masked self attention builds context from the output prefix
2. cross attention brings in encoder information
3. the feedforward network transforms each token's features
residual connections & normalization surround them. stacked blocks refine the representation, then a linear layer maps it to vocabulary logits. softmax gives token probabilities.
the original feedforward layer expands width from 512 to 2048 and projects it back again.
the separate components are now one path from the output prefix to the next-token distribution.