Archi

CloudNative Engineer ☁️ | Interested in Ethereum core protocol 🔮 | Rustacean 🦀 | Gopher | Zig | Neovimmer

[Archi @__noob__coder__]

Best lesson I learned from @tsoding 😉

[Daniel Moka⚡ @dmokafa]

A senior developer in one stack might be a junior in many others.

But their expertise is not defined by their stack.

It is defined by their language-agnostic mindset.

If someone develops this mindset, picking up a new language or framework takes only a couple of weeks.

[Mischa van den Burg @mischavdburg]

Only 3% of US consumers pay for AI

When you're in the bubble it seems so natural, but most people don't have a clue

If you pay for AI you're already WAY ahead of the curve

[Darryl Ruggles @RDarrylR]

Kubernetes does not delete Persistent Volume Claims (PVCs) when their pods go away. This helps to protect the data but can leave orphaned volumes. Determining whether a claim was still used by anything involved cross-referencing many things, but now in Kubernetes 1.37, the PVC protection controller adds an "Unused" condition to PVCs.

The condition tells you when no pod references a PVC. It's not telling you the data is safe to delete. You have to determine that yourself. Check this out from Roman Bednář.

https://t.co/H5g58AeWOw

#Kubernetes #Storage #FinOps #PlatformEngineering

[TensorTonic @TensorTonic]

the best way to learn inference is by implementing it yourself

https://t.co/rwSSNnQe4I

[Daniele Polencic — @[email protected] @danielepolencic]

Kubernetes hard mode: how do banks run their platforms?

My new book, Kubernetes Architecture in Financial Services, is out now:

https://t.co/pCV2jpyX2C

Here is what surprised me while researching it:

[Jorge Castillo @JorgeCastilloPr]

The Chinese already killed Jev. This one went fast

[Arpit Bhayani @arpit_bhayani]

btw, I also run an engineering "newspaper" curating blogs, articles, and papers on system design, databases, and applied AI.

I keep it updated every day. It publishes with a 2-day delay, but is heavily curated.

It is called https://t.co/IyLtJNHLWy, you might find it helpful. High signal, low noise, always :)

[Shubham Mishra 🐐 @shubh6200]

My personal favourites :
1. https://t.co/tYQwV36GdN
2. https://t.co/VWjwKvpH55
3. https://t.co/0b8pCMPP1X

[Jaana Dogan ヤナ ドガン @rakyll]

🥳 Excited to start revealing what we've been working on in the last few months. First, we decided to reinvent Kubernetes for agentic workloads with statefulness and fast resumption. Secondly, we are building an agentic orchestrator that will be Google's open agentic orchestrator and runtime. https://t.co/XCrdAVvDAe

[Ayaan 🐧 @twtayaan]

Google just dropped new Kubernetes infrastructure built to run MILLIONS of AI agents.

And they open-sourced the whole thing.

This is Agent Substrate, and it is now available on GKE.

Instead of keeping thousands of idle AI agents sitting around consuming CPU and memory, Agent Substrate can suspend them, release those resources, and bring them back in milliseconds when needed.

Here is what it does:

→ Runs 10x more agent environments on the same compute compared to standard container runtimes
→ Suspends idle agents and resumes them in under 500ms while preserving their full state
→ Supports 500+ agent suspend/resume operations per second
→ Packs 1,000+ dormant agent environments onto a single host
→ Runs agents inside isolated microVMs or gVisor sandboxes for stronger security
→ Works with Kubernetes APIs and can run on GKE or other Kubernetes clusters

AI agents spend most of their time waiting for models, tools, APIs, or humans. Keeping compute reserved while they wait gets expensive fast.

Agent Substrate suspends idle agents, frees those resources, and restores them when needed.

Google is basically building a Kubernetes-native execution layer for running AI agents at massive scale.

Announcement here:
https://t.co/ZQmNRlWjni

[Rogerio Chaves @_rchaves_]

A programming language that:

- Runs fast like C
- Compiles fast like Go
- Garbage collects like Rust
- Typechecks like Haskell
- Proves like Lean
- Compiles to GPU

Is then the last programming language we’ll ever need, AI age or not

Extremely bullish on Bend2

[Suraj Sharma @suraj_sharma14]

As an AI Infrastructure Engineer, you must build these projects.

Systems that prove you can serve, scale & keep GPUs honest.

1.) Self-Hosted Inference Cluster
Build: Multi-node GPU cluster serving an open model with vLLM or SGLang, health checks, and a public latency report.
Why: Talk is cheap. A live cluster is the portfolio.

2.) Cost-Per-Token Dashboard
Build: TTFT, ITL, GPU util (DCGM) and $ per 1M tokens by model, tenant and route.
Why: Infra without FinOps is just expensive uptime.

3.) Queue-Based GPU Autoscaler
Build: KEDA (or equivalent) scaling on queue depth, with cold-start mitigation and spot fallback.
Why: Idle GPUs kill startups. Slow scale kills users.

4.) Continuous Batching Load Test
Build: Traffic generator that stresses continuous batching, KV cache limits and saturation points.
Why: You don’t know your serving stack until it breaks under load.

5.) Model Weight Delivery System
Build: Safetensors registry + sharded weights + CDN/lazy loading for fast node bring-up.
Why: Cold starts are often a storage problem pretending to be a compute problem.

6.) Multi-Model AI Gateway
Build: Routing, retries, fallbacks, rate limits and per-tenant token budgets across 2–3 serving backends.
Why: Production isn’t one endpoint. It’s a traffic policy.

7.) Quantized Serving Bakeoff
Build: Same model in FP16 vs AWQ/FP8; publish quality vs latency vs VRAM tradeoffs.
Why: Optimization without benchmarks is cosplay.

8.) Secure Multi-Tenant Inference Layer
Build: Tenant isolation, keyed access, sandboxed tool/exec paths, audit logs.
Why: One noisy neighbor (or leak) ends the B2B deal.

9.) Checkpointed Distributed Training Job
Build: Ray or Spój/Slurm job with FSDP/tensor parallel, fault injection, and resume-from-checkpoint.
Why: Training infra is judged on recovery not the happy path.

10.) Speculative Decoding Prototype
Build: Draft + target model path with prefix caching / chunked prefill; measure accepted-token rate.
Why: Speed wins are product features when you own the stack.

11.) GPU Partitioning Lab
Build: MIG (or equivalent) slices with fair scheduling, resource limits, and contention tests.
Why: Real clusters share silicon. You must prove fairness under pressure.

12.) Triton / Custom Kernel Path
Build: One hot path accelerated (attention, sampling or tokenizer-adjacent) with before/after metrics.
Why: Senior infra energy = knowing when frameworks aren’t enough.

13.) Observability Spine for Non-Determinism
Build: Traces for queue time, prefill, decode, cache hits, errors alerts on drift and cost spikes.
Why: You can’t page what you can’t see.

14.) Multi-Region Failover Drill
Build: Active-passive or active-active inference with DNS/failover and RTO/RPO notes.
Why: “We have GPUs” isn’t reliability. Failover is.

15.) Public Benchmark Teardown
Build: Publish an architecture + latency/cost report others can reproduce.
Why: Hiring managers trust public artifacts more than résumés.

Most people watch tutorials. Builders ship systems.

Bookmark & Repost.

[Archi @__noob__coder__]

@arpit_bhayani NeoVim 😈

[William Gilpin @wgilpin0]

Our group discovered that reasoning models produce fractals when asked to solve hard problems. We can use nonlinear dynamics to probe the thinking processes of recurrent depth models on Sudoku, mathematics, and even ARC-AGI (1/N)
https://t.co/Q3u8OqylZf

[Mrinal @Hi_Mrinal]

Interesting read on internals of cloudflare workers module registry liked the webassembly source phase imports part coz it was pretty new thing for me :D

https://t.co/SDCDEg5Zqu

[Archi @__noob__coder__]

@ChShersh @mertonesque There is a concept in Rust called : TT munchers 😉 .

[Nihal Pasham @npashi]

🎉 CUDA Rust is here! Team 🟩 ❤️ 🦀

You can now write CUDA kernels in plain Rust, two ways: cuda-oxide (SIMT) and cutile-rs (Tile).

https://t.co/e4AQHUvskP

[Jakob @Jakob_D04]

Ich habe mir mal genauer angeschaut, welche #AfD-Abgeordneten in den neuen Landtag von #SachsenAnhalt einziehen werden, abseits der Bekannteren:

[Hedgie @HedgieMarkets]

🦔An NYU mathematician says OpenAI used his own progress against him to beat him to one of the biggest unsolved problems in mathematics. Tristan Buckmaster had been working toward a Millennium Prize proof using OpenAI's Codex when information about his progress reached OpenAI.
Days later, OpenAI published a full proof of the same problem using the same uncommon approach, after burning $22.5 million in compute to get there. When Buckmaster confronted them, exec Sébastien Bubeck allegedly said "Why would you ruin your career?" and "If you don't want me to be nice, then I don't have to be nice."

My Take
OpenAI spent $22.5 million to solve a problem with a $1 million prize. They didn't do this for the bounty. They needed a headline that says "our AI solved one of the hardest problems in mathematics" and they needed it before someone else got credit. They started days after they heard about Buckmaster's progress and took the same uncommon approach he'd pursued for months. That's hard to explain as coincidence.

Buckmaster did his work inside Codex. OpenAI reserves the right to train on Codex data. They admit they can't rule out that his usage helped improve their models. So a customer used their product, potentially handed them the roadmap, and then OpenAI outran him with $22.5 million in compute he could never match. I don't know if any of this was intentional. But if you're a researcher and you just watched this happen, I don't think you'd keep your best ideas inside someone else's product.

Hedgie🤗

https://t.co/1YnggU7xdI

[Arpit Bhayani @arpit_bhayani]

Quantization is an essential piece in inference engineering, and I just made and published a video covering its fundamentals...

- what quantization is
- why it is required in the first place
- what model weights actually are
- and how all of this makes inference faster and cheaper.

For example, Llama 3.1 has 405B parameters. At a classic 16-bit representation, we would need roughly 810 GB just for the weights.

The gist is: quantization kicks in to optimize memory requirements and make inference faster and more efficient by reducing 16-bit floating-point numbers to 4-bit integers; the theoretical size drops to roughly ~200 GB.

Still huge, but suddenly something that was nowhere near practical becomes much more approachable. How does this actually work?

Give it a watch.

[Archi @__noob__coder__]

Spoiled my early morning mood!

[Arnaud Bertrand @RnaudBertrand]

This sounds completely unreal but I double checked and it's true: the Eiffel Tower was closed on Monday due to a Hindu sect that demanded that all women "make themselves invisible" so their 93-year-old guru wouldn't have to see them.

The sect in question is BAPS - "Bochasanwasi Akshar Purushottam Swaminarayan Sanstha," which roughly translates to "the organization of Swaminarayan followers who believe in Akshar Purushottam, based in Bochasan" (a village in Gujarat).

It's a very wealthy organization with deep ties to Modi's BJP (Modi is himself from Gujarat), known for building mega-temples around the world and whose doctrine demands absolute obedience to a living guru as the sole path to God.

It's not their first controversy: in 2021, the FBI raided their massive New Jersey temple after workers - mostly Dalits, India's lowest caste (the so-called "untouchables") - alleged they'd been lured to the US on fake religious visas, had their passports confiscated, and were forced to work 13-hour days for about $1 an hour, confined to a guarded compound (see https://t.co/kTNzdZWtoe).

Long story short, they just opened a new mega-temple in France (the website: https://t.co/4g6HW9bBPy) - specifically in Bussy-Saint-Georges (30km away from Paris) - presented as "the biggest Hindu temple built in continental Europe in a traditional style" (https://t.co/exbFjzLe2k).

Their guru was there for the inauguration and, as you do, wanted to visit the Eiffel Tower while in France. But, thing is, his doctrine dictates that women must not be in his presence.

And here is where it gets unreal: when his entourage requested a private visit, the Eiffel Tower's management actually agreed to the terms and ordered all female employees - including subcontractors - to hide (https://t.co/Sm7yb5LMFq).

This was on Saturday and, France being France (in a good way), on Monday the women who'd been told to make themselves invisible made themselves very, very visible: the staff went on strike and shut down the Eiffel Tower.

So there you have it, unreal but true: the Eiffel Tower, symbol of the French Republic, is closed because a 93-year-old Hindu guru refuses to be in the presence of women.

(picture below the picture of their guru on his visit, taken from https://t.co/4FRKezWJvq)