Hugo M.

Programmer @zml_ai . Prev. Preligens (acq. by Safran), Station (YC W18, eF17)

[Steeve Morin @steeve]

another 5 days later, zml/llmd runs fully on Metal, serving 8 simultaneous requests at full bf16

zml/llmd is our LLM serving software that all the modern niceties (continuous batching etc..)

ps: "full bf16", what a time to be alive

[Steeve Morin @steeve]

Qwen 3.5 running on Amazon Inferentia 2 with @zml_ai, perf on the low side, but it works

[Hugo M. @hugomano]

@MainzOnX One of the best abstractions I’ve ever used

[Steeve Morin @steeve]

it seems AWS has cooked in their new SDK release

[Corentin Kérisit @corentinanjuna]

Hermetic LLVM now supports cross compiling libstdc++ from source.

Community asked for it and it was a prerequisite for supporting hermetic remote CUDA compilation since CUDA prefers libstdc++ headers over libc++.

Another step towards 100% hermetic builds.

[ZML @zml_ai]

Introducing zml-smi, a universal diagnostic and monitoring tool for GPUs, TPUs and NPUs.

Real-time insights into the performance and health of your hardware.

Read & Download: https://t.co/zFUPLj2Sh5

[Steeve Morin @steeve]

We managed the holy grail of CUDA compilation: joining CUDA host code and device code at *link time*.

This means both build graphs (device and host) are now completely separated and built by SM.

True scalable CUDA build graphs are now possible. Those who know, know.

[Corentin Kérisit @corentinanjuna]

sitrep: casually cross-compiling flash attention using CUDA 13.2 and llvm@HEAD from my macbook pro using remote exec feels unfair.

[Hugo M. @hugomano]

This rewrite was intense, but it unlocks a completely different level of control and performance for anyone serious about inference systems.

We flipped the model: explicit over implicit. Composability everywhere. Performance as a consequence of design, not patches.

[ZML @zml_ai]

Introducing ZML/v2

ZML/v2 is a complete rewrite of the framework with a focus on developer experience, performance and most importantly composability.

Link in next tweet.

[UMA @UMA_Robots]

UMA is launching to build general-purpose mobile and humanoid robots from Europe.
Join us: https://t.co/WE6R7YHDKC

[Frank @jedisct1]

I’m organizing a friendly meetup about performance in Paris on December 4. Whether you’re tuning databases, shaving CPU cycles and packets, optimizing AI inference, building web services, or simply curious about making things go faster, join us!
https://t.co/j3Dk7Lw9XU

[Hugo M. @hugomano]

In the Bay Area Nov 15–22 with part of the @zml_ai team—@steeve @TheCorendos. Happy to chat inference, production model serving, compilers and chips. Ping me if you’re around!

[Corentin Kérisit @corentinanjuna]

Take a moment to reflect on what it means :) 🔥

[Steeve Morin @steeve]

Paving the way for unlimited context windows.

Introducing @zml_ai /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

[Steeve Morin @steeve]

This is the first time we're going to talk about our new tech, would love to see you all there

[Steeve Morin @steeve]

Sex is good but have you tried loading tensors from a SSD to a 4090 at 15GB/s ?

[Steeve Morin @steeve]

I'll be presenting our work on sparse logarithmic attention (over UDP!) at @aiDotEngineer on September 24th.

[Hugo M. @hugomano]

@StijnSmits @steeve Really interesting! If you want to dive deeper, check this out: https://t.co/PvETNZn69N and https://t.co/vqo6icXpyo.
Logical mesh and physical mesh are distinct so aligning them is key to maximizing TPU utilization (cause of diff. bandwidths). I’ll admit, it’s a bit of an art :)

[Hugo M. @hugomano]

There’s something deeply satisfying about shaping the full inference path — from low-level kernels to object files, from ABI quirks to the final binary — and knowing every piece is yours to understand, patch, and rebuild.
It’s not just control. It’s joy.

[Erik Kaunismäki @ErikKaum]

The @zml_ai team and @steeve cooked a new high-performance LLM inference engine.

- lightweight 2.4GB container
- easy on cross-platform
- written in zig

I just tested it and deployed on @huggingface Inference Endpoints. You can try it out in 5min! 👇