Tin Rabzelj

Building cool things.

Rust, TS/React, AI 🚀

[Tin Rabzelj @tinrab]

If you're stressed, burned out, and need to loosen up, consider building your own operating system. #rustlang

[Tin Rabzelj @tinrab]

I was reading blog post "How we achieved truly serverless GPUs" by Modal (https://t.co/CQ39MppPwG). The engineering is pretty insane. I came across "CUDA checkpoints," which I somehow never saw before. They can be used alongside CRIU to checkpoint and restore process' state from disk.
So I added it to my Rust CUDA crate.

[Tin Rabzelj @tinrab]

API design for my NCCL Rust wrapper (not final).
Creates device group with one rank per device, creates send/recv buffers, runs all-reduce (take every rank’s input buffer, sum element-wise across ranks, and write the same result to every rank). It hides direct calls to `ncclCommInitAll`, `ncclGroupStart`, etc., and hopefully makes it easier to use in a correct way.

[Tin Rabzelj @tinrab]

Blog post: https://t.co/O6yMIX7hqK
🦀 crate: https://t.co/YqIeAytVe3

#rustlang #cuda #nvidia #nvml

[Tin Rabzelj @tinrab]

I've built a crate for using NVIDIA Management Library (NVML) in Rust.

Safe and raw bindings. FFI bindings have documentation comments that are near-identical to official docs.

[Tin Rabzelj @tinrab]

@threepointone My top 5 aren't even on the list. Google being so high is sus.

[Tin Rabzelj @tinrab]

@zmzlois I didn't either. I don't know why we need more words to describe something that calls an API.

[Tin Rabzelj @tinrab]

@boshen_c I actually thought about porting it, but too busy porting some other stuff currently 😀 From the benchmarks I've seen, it seems it would be a big W.

[Tin Rabzelj @tinrab]

@jarredsumner @LukasHozda I assume you're planning to do miri and clippy lints, and run with an agent in a loop until it fixes everything.

[Tin Rabzelj @tinrab]

@joshmo_dev Lol. I wish "too employable" was true. There are like 20 Rust jobs.

[Tin Rabzelj @tinrab]

I thought about something similar years ago. I even built a "unified language compiler" and a runtime (as a wasm target) that you can hook into however you want. Kind of like a universal plugin system. It could turn on/off settings into a full programmable flow graph (similar to Unreal Engine's blueprints) and also change ui. The problem is, it's not clear what kind of users are the target audience of something so extremely extensible. I have some ideas, but not sure in what way it'd be worth it.

[Tin Rabzelj @tinrab]

@FrancescoCiull4 @jarredsumner @bunjavascript I'm waiting for when/if the number of crash reports drops.

[Tin Rabzelj @tinrab]

@theo By the way, `unsafe` keyword does not disable the borrow checker. https://t.co/NZ9N0OY41S

[Tin Rabzelj @tinrab]

@theo I keep hearing about CVEs but nothing about anything bad happening.

[Tin Rabzelj @tinrab]

I ported Nvidia's cuDNN frontend library from C++ to Rust. I needed it in Rust and wanted more control over the internals. cuDNN-frontend provides a simpler graph API over the low-level cuDNN backend. You build a graph of operations, and it handles the lowering to backend APIs, kernel fusion, autotuning, version compatibility expansion, virtual tensor tagging, etc.
I also added more features like a nicer way to interleave custom kernels on the same CUDA graphs as cuDNN. It’s still incomplete and not open source yet.

Here's a snippet from the official examples, NVFP4 matmul and FP16 SDPA, and my own version (not final):

[Tin Rabzelj @tinrab]

LaTeX math in Rust docs for my CUDA crates.
Not final; getting Markdown docs ready is a bit finicky.
#rustlang

[Tin Rabzelj @tinrab]

Inferring types from CUDA kernels in Rust.
API isn't final.

[Tin Rabzelj @tinrab]

@theo That makes it worse. The amount of engineers freaking out about 10-20% better model is crazy. Like, there literally won't be anything novel about it.

[Tin Rabzelj @tinrab]

@HSVSphere "It was really hard to *generate* good software."

[Tin Rabzelj @tinrab]

@yacinelearning These are all over my timeline. It definitely reads like it was LLM-generated. Idk, it's probably just regular engagement bait.

[Tin Rabzelj @tinrab]

@theo @ssijak Is this about the npm scripts, "clean" being overridden in package.json? I think the obvious correct choice is to clearly separate built-in commands from npm scripts (always run with "npm run ..."). Implicitly changing behavior project-to-project seems bad.