nuxyel

AI & ML Engineering R&D @samsung
Building ML in Bend 2

[nuxyel @nuxyel]

BendTorch is out: neural networks trained in Bend 2, with the backward pass in the types. MNIST, a small Llama and SmolLM2-135M fine-tuning, each checked against a PyTorch twin from the same weights.

Every gradient has the type of the value it is the gradient of, so a transposed gradient or two layers that do not fit do not compile. The backward formulas of the linear layers, the chain rule through Seq and Fork, and the causal mask are laws that Bend's kernel checks.

It computes what PyTorch computes: MNIST's loss and its 9,108 test hits are PyTorch's, with the same bits on 1 and 16 threads. After 400 fine-tuning steps on Shakespeare, SmolLM2-135M writes "And her love is in the bosom more than thou art in me", and greedy decoding gives PyTorch's tokens. Of 98 bugs I planted on purpose, 96 are caught by a law or a test; the other 2 still compute the right result.

It is slower: about 6x PyTorch eager on MNIST and 4.3x on a SmolLM2-135M training step on 16 threads. It is faster only on a tiny model, where PyTorch's per-operation overhead dominates. On the GPU it is 1.7x slower than 16 CPU threads. BENCHMARK.md compares eleven systems, losses included.

All of the math is Bend: no BLAS, no C or CUDA kernels. It builds on bend-ml and on what llama.bend taught me about the Llama layers, and it is on BendHub as four packages ([email protected], plus -fast, -gpu and -load).

Cc @VictorTaelin

https://t.co/oCk5zbdLtA

[nuxyel @nuxyel]

guys I really dont know why but everytime I hand-write code drunk I feel like im the best programmer ever.

[nuxyel @nuxyel]

has anyone built, or started building, a database in Bend? queries checked against the schema at compile time sounds like a good fit. also a DELETE or UPDATE without a WHERE just wouldn't compile xD

[Kevin Ngo @kevin_t_ngo]

Just found out about Bend, so I had Claude make an explainer on why agents might prefer it

Every frame was drawn in Bend. Claude built a custom renderer in Bend to make the video

[nuxyel @nuxyel]

I'm still deciding what my next project will be. Most of the ideas I have are fairly big, and I'd rather take the time to pick the right one than spend months on the wrong one.

Meanwhile I'll keep improving the three I have: bend-ml, llama.bend and BendTorch. Right now that means moving them to Bend 2.0.38.

And just to be clear, yes, the next one will be in Bend too. I really believe in where the language is going, and I want to keep contributing to it as much as I can.

[nuxyel @nuxyel]

@VictorTaelin some corno its just like this vibe

[nuxyel @nuxyel]

nice, catching shape errors on a laptop before you touch a GPU is the right instinct. we got there differently... here the shapes live in the types, so a mismatch is a type error, and the backward formulas are laws the kernel checks. the s-expression experiment files are a cool idea. thanks for sharing, I'll take a look for sure

[nuxyel @nuxyel]

BendTorch is out: neural networks trained in Bend 2, with the backward pass in the types. MNIST, a small Llama and SmolLM2-135M fine-tuning, each checked against a PyTorch twin from the same weights.

Every gradient has the type of the value it is the gradient of, so a transposed gradient or two layers that do not fit do not compile. The backward formulas of the linear layers, the chain rule through Seq and Fork, and the causal mask are laws that Bend's kernel checks.

It computes what PyTorch computes: MNIST's loss and its 9,108 test hits are PyTorch's, with the same bits on 1 and 16 threads. After 400 fine-tuning steps on Shakespeare, SmolLM2-135M writes "And her love is in the bosom more than thou art in me", and greedy decoding gives PyTorch's tokens. Of 98 bugs I planted on purpose, 96 are caught by a law or a test; the other 2 still compute the right result.

It is slower: about 6x PyTorch eager on MNIST and 4.3x on a SmolLM2-135M training step on 16 threads. It is faster only on a tiny model, where PyTorch's per-operation overhead dominates. On the GPU it is 1.7x slower than 16 CPU threads. BENCHMARK.md compares eleven systems, losses included.

All of the math is Bend: no BLAS, no C or CUDA kernels. It builds on bend-ml and on what llama.bend taught me about the Llama layers, and it is on BendHub as four packages ([email protected], plus -fast, -gpu and -load).

Cc @VictorTaelin

https://t.co/oCk5zbdLtA

[nuxyel @nuxyel]

yeah, and it holds bit for bit, not just in exact math.
btw the row sum is one fixed pairwise tree, and a block is a subtree of it, so changing the block size only changes which pass computes which node. encoding "the block divides the row" in the type is a nice idea and fits how the rest is built. noted, I'll try it

[nuxyel @nuxyel]

@SubtleGradient @leastsquared_ yep, that's my favorite part btw. Linear(784,128) followed by Linear(64,10) is a type error before anything runs

[nuxyel @nuxyel]

@jimduey thank you!

[nuxyel @nuxyel]

@Nipsuli thanks bro!

[nuxyel @nuxyel]

thanks! it isn't a tape-based autodiff over arbitrary code. each layer carries a typed backward pass, and the combinators (Seq, Fork, Both, Residual) apply the chain rule, so any model you compose from layers gets its backward for free. a new primitive needs its backward written by hand, and the laws check that it's right

[nuxyel @nuxyel]

type-checking is cheap: the tiny Llama (~80 composed layers) checks in 0.6 s, and the shapes are erased, so they cost nothing at run time. the expensive part is building: Bend's C output plus clang takes about 40 s for the tiny Llama. I haven't timed the SmolLM2 build on its own. PyTorch eager has no build step at all, so that's a real cost on my side

[nuxyel @nuxyel]

good question. neither is about the shapes I happened to test. one makes a one-cell array, so every index wraps to 0 and the sum comes out right by accident. the numbers miss it, but a bounds check catches the same fault, so a bigger random shape would catch it too. the other only changes a GPU block size, and any block size that divides a row gives the same sum, so that one is a true equivalence

[nuxyel @nuxyel]

@LouisMullie thank you so much!

[nuxyel @nuxyel]

@jotaporrasv @VictorTaelin damn. saving it for the CUDA version 🔥 cant believe I missed that one lol

[nuxyel @nuxyel]

@zxv_writes thanks! yes, a Bend-written compiler is on the roadmap. lots of room: on the GPU most of a step is runtime overhead, not math (e.g. bendlang/bend#1776, up to 20x on tight CUDA loops). I'll rerun the same benchmarks on every release and post what moves

[nuxyel @nuxyel]

@elffholic thx!

[nuxyel @nuxyel]

@kazzzz520 thanks man, means a lot! bend-ml's row bands started from your Bend-Conv post, so Stelliferous is in here too!

[nuxyel @nuxyel]

on CPU it's ~4.3x slower than PyTorch eager for a SmolLM2-135M training step on 16 threads (4.0 s vs 0.93 s), ~6x on MNIST.
vs torch on CUDA it's way behind for now: the same step is 126 ms in PyTorch on my RTX 4050 and ~7 s in BendTorch's GPU path, which is even slower than 16 CPU threads. most of that time is Bend's GPU runtime rather than the math, and HOC says it's about to get much faster. full table with 11 systems: https://t.co/1HicmdGXHN

[Taelin @VictorTaelin]

people building cool things in bend. I should react liek "cool, adoption" but I'm more like "fuck, people are using it"

I do think Bend is good, but the plans I have will make it GREAT. it will be astronomically better soon, and that's what I want people to experience