Moncef Abboud

[Moncef Abboud @moncef_abboud]

@fchollet The steps of some business workflow ? Sure, understanding inventory() and ship() is doable.
But pure technical stuff, infra? K8s? DB? authz/authn?
Without practice, very hard to build understanding. Then what ? Have an agent read the logs and metrics you don't even comprehend.

[Moncef Abboud @moncef_abboud]

@wafer_ai @AMD 196 tok/s/GPU at 172 tok/s/user on B300. That's high!
vLLM's GB300 Pareto curve hits ~1000 tok/s/GPU at 40 tok/s/user, and drops at its best to ~105 tok/s/user

https://t.co/cdj1eQb5BD

[Moncef Abboud @moncef_abboud]

How Profitable is LLM Inference? Doing the Math on Kimi K3
https://t.co/lhHF6l3seV

[Moncef Abboud @moncef_abboud]

Exploring Speculative Decoding
https://t.co/Djazq4MgA5

[Moncef Abboud @moncef_abboud]

Deep dive into ideas from the excellent @vllm_project: KV cache, paged attention, tensor parallelism.

https://t.co/QO3I7vJ01c

[Moncef Abboud @moncef_abboud]

@eatonphil Nice write up.
Copying from Stackoverflow usually forces you to understand and adapt a snippet. It often breaks and forces digging. Pasting into an LLM/using an agent often skips that friction entirely so the need for understanding can be way smaller. Your point still stands.

[Moncef Abboud @moncef_abboud]

@arpit_bhayani Nice write up. For values, Prometheus does something similar by storing the XOR of the current sample value with the previous one => lots of zeros since sample values are very close to each other. And a single 0 bit if they're exactly the same (XOR is 0).
https://t.co/fSJGgb5ubd

[Moncef Abboud @moncef_abboud]

The new inplace restart is way faster and thus more economical.
Crazy how something this seemingly simple can have such an impact.
More details: https://t.co/ZFrVzpMTcR

[Moncef Abboud @moncef_abboud]

Kubernetes v1.35 introduced a simple new feature that saves over $100,000 for a 1k nodes cluster.

What is it?

[Moncef Abboud @moncef_abboud]

"Restart All Containers" in a pod.

If a container in a large ML job fails, all pods need to be restarted.

Prior to this feature, there was no way to do that other than deleting and recreating the pods. This takes time. Pretty expensive when you're using many GPUs/TPUs.

[Moncef Abboud @moncef_abboud]

@jayair Did a deep dive into Opencode’s repo before the move to Opentui (a few months ago), and the code quality and architecture were really a hoot. Kudos to the team!
https://t.co/spokwQ2ZTl

[Moncef Abboud @moncef_abboud]

@0xlelouch_ https://t.co/9NI5lOXpnr Kafka config property defines the max time a connection can be idle. Setting it to a value lower than the firewall's limit should prevent abrupt disconnections and allow the consumer/producer to re-establish the connection when needed

[Moncef Abboud @moncef_abboud]

https://t.co/AYiJpcYIFy

[Moncef Abboud @moncef_abboud]

It's not always DNS!

[Moncef Abboud @moncef_abboud]

https://t.co/DnbVuWli9w

[Moncef Abboud @moncef_abboud]

Wrote something about TCP

[Moncef Abboud @moncef_abboud]

Just to add some context if you’re wondering what’s happening.
Computers are identified via IPs (like 172.66.0.227), since those are hard to remember, we use domain names as aliases ( https://t.co/2kwaEWlk8Z -> 172.66.0.227).
if you open your terminal and enter `dig +short https://t.co/2kwaEWlk8Z`, you'll get that IP.
When you visit https://t.co/5pOMmYQ2dy, your browser asks a DNS resolver like 1.1.1.1 (Cloudflare) or 8.8.8.8 (Google) to resolve that name to an IP.

Resolver reach out to DNS servers that store DNS data.
DNS is hierarchical: there are root/seed servers that know about TLDs (.com, .io, etc.) which in turn know where domains (https://t.co/2kwaEWlk8Z, https://t.co/VFMNdS9EJM) live and each domain has authoritative name servers that store the actual records such as A/AAAA (a type of record) for name to IP mapping. There are other types of record (mx for email, cname, etc)

In the example above, both resolvers (1.1.1.1 and 8.8.8.8) reached AWS’s authoritative name server (https://t.co/cJQuVVhx8X) but it didn’t return an IP for the Dynamodb name. Basically the chain worked, but the final mapping (domain -> IP) was missing (that's the crux of the issue, why was it missing?).
Anyway, without knowing the IP of Dynamodb, other services were unble to connect to it.

[Moncef Abboud @moncef_abboud]

LLMs and MCP often suffer from context bloat.

Experimenting with dynamic context loading: a minimal loader tool summarizes available capabilities, and the LLM loads specific tools only when needed.

https://t.co/5KSYRHwzU3

[Moncef Abboud @moncef_abboud]

Exploring Terminals, TTYs and PTYs.
https://t.co/7JgwXVgMBx

[Moncef Abboud @moncef_abboud]

Kudos to @thdxr @adamdotdev and the other contributors for a great product!

[Moncef Abboud @moncef_abboud]

Spent some time exploring Opencode’s internals. Super fun!
https://t.co/spokwQ2ZTl