b/acc, context platform engineer

AI Factories. Balanced Accelerationist. WEKA CAIO, CNCF kubernetes founding board, Post-PKI.

[b/acc, context platform engineer @AccBalanced]

Why do you need a Cyber Defense Factory?

As expected, we’ve matured from a post-Mythos/Glasswing cyber reality, to a post-Black October cyber apocalypse.

Your AI infra isn’t ready for this
Your cloud’s AI infra isn’t ready for this

@deepseek_ai's DeepInfra architecture based on a centralized KV cache pool on their 3FS storage system, is one of the leading AI infrastructures that is. Yet there’s one even better.

As measured most recently by the @nebiusai TokenFactory, Memory beats Storage for the KV cache critical path of agents by at least a factor of 3. Based on @weka 's Augmented Memory Grid, @EmbeddedLLM's TokenVisor inference software and service, is the first Agent Factory capable of rapidly and efficiently serving 24/7 agent swarms on a multi-silicon platform, featuring AMD.

That is the core of what’s needed for a Cyber Defense Factory.

DeepSeek, Nebius and EmbeddedLLM presently operate the only agent swarm architectures able to immediately, sustainably overmatch and defeat attacks from adversarial cyber agent kill chain swarms.

If you manage or subscribe to an AI SOC, contact EmbeddedLLM and WEKA immediately to gain back your cyber advantage.

(credit to @edsim for the image, and coining the ‘Cyber Defense Factory’ term, and @epowell101, for pioneering AI SOC innovation)

[b/acc, context platform engineer @AccBalanced]

@Jo1uck only a few have seen that wall ;)

[b/acc, context platform engineer @AccBalanced]

@pingToven @teortaxesTex but cache for how long? (cc: @tarunchitra )

https://t.co/hf6hdUC457

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

And OpenRouter is the worst way possible to consume Claude tokens (you don't even get cache discount and you pay their psychotic API margins). So it's a *LAGGING* indicator of Anth's revenue surge. They're probably 2x ahead of OAI right now.

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

Now that's more like it. K3.1 is between 6 Sol and 1017.
Kimi was earlier to high quality vision, it's much bigger, it should perform well on robotic/spatial/agent tasks…
the issue is, again, that it costs 20 times more.
Sol 6.1 must be the north star of Chinese models now.

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

Confirmation of K3.1 Flash I guess

[Woosuk Kwon @woosuk_k]

We wrote a blog post on how we achieved such great performance on DSV41-Flash. Our results are also verified in the @SemiAnalysis_ AgentX benchmark.

[b/acc, context platform engineer @AccBalanced]

@_ram_ Usually cyber insurance is the ultimate catalyst for action by CISOs and their CIO/CFO business partners. But this crisis won’t wait for denial or premium increases

[unusual_whales @unusual_whales]

For the first time ever, gas-only cars make up less than half of new vehicles sold worldwide, per Bloomberg.

[GDP @bookwormengr]

Makes sense. Imho, won't degrade performance.

The issue with large scale up bandwidth is that it does not get used all that much in NVL72 kind of racks. For example, in DP-EP configuration it is used to move around activations from DP to EP and vice a versa. It is very little data compared to bandwidth available even for large batch size.

The only time it gets used aggressively is in Tensor parallel, but if scale up domain is large, people rather deploy DP-EP rather than DP-TP.

Even in a scale up domains, most bandwidth consumption is GPU-HBM for reading weights and KV cache. GPU-GPU NVLink consumption is very low unless using TP. These two are different pools of bandwidths; but numerically speaking former gets used till saturation and while later is used much less. Training does use lot of scale up bandwidth, but even there it is not saturated.

For scale up domain, the number of GPUs/NPUs/ASICs you are able to connect with low latency connection is much more important than sheer bandwidth (some literation call it 'world size').

[Zephyr @zephyr_z9]

Nvidia also reduced the NVLink bandwidth per GPU from 1.8 TB/s (3.6TB/s bidi) to 1.5 TB/s (3 TB/s bidi)

AMD now has a higher HBM bandwidth (23.3 TB/s) as well as a higher scale-up bandwidth (1.8 TB/s)

[Christian Keil @pronounced_kyle]

@zackslab we are currently just above the "funny robot can do monkey tricks" stage for hardware design

[Vikram Sekar @vikramskr]

BoA calculates ~2.7K CPUs/1M users which lines up pretty much with my own calculations.

But 21M CPUs in 2030 per billion users is a BS number.

They assume 8-16 vCPUs dedicated allocations (100% utilization) per user. Cloud CPU deployments never allocate 100% CPUs for a billion users.

This BS number will then be recirculated in other reports ad nauseum.

[b/acc, context platform engineer @AccBalanced]

@jtatarchuk Hey Jeff - our team @EmbeddedLLM can use them all with @weka AMG. We will follow up via email.

[b/acc, context platform engineer @AccBalanced]

Great. Please do let me know how you prefer to learn, and I will respond.

Meanwhile - Here's the intro to an objective long read on this subject, which you can dissect with your chatbot. It's penned by the highly respected and followed researcher @Machine01776819
https://t.co/06H8EzNbg7

If you prefer, your fav chatbot can follow these newer breadcrumbs to see how 24/7 persistent agent swarms are now affordable and fast:

1. Memory is widely acknowledged as the bottleneck to inference
2. DeepInfer has proven you can use flash storage as non-volatile memory to extend KV cache, and efficiently scale agent swarm performance to 3 days, with chat sessions to 7 days
2a. Note most inference providers today scale inefficiently, adding more GPU servers just for the needed HBM+DRAM
3. WEKA's Augmented Memory Grid (AMG) inspired DeepInfer, with my team posting about this KV cache storage approach since Feb 2025
4. AMG uses high-bandwidth networking (NVLink, UMBP) to enable storage capacity as high-bandwidth memory, combining the best of both, scaling efficient agent performance to weeks and months
5. Most recently ahead of their successful Token Factory cached token rollout, Nebius published updated benchmarks showing WEKA AMG processing 2-4x more requests and input/output tokens. Previously CoreWeave, Firmus and Oracle published similar AMG benefits with frontier models current at that time.

Finally If interested, I can DM you some detailed interactive dashboards which outline the tech in an even more engaging way.

[Atharva Vaidya @atharvavaidya]

The Twitter bubble is so hilariously ahead of everyone else it's a joke... not surprising at all!

[Annika Lewis @AnnikaSays]

The best decisions I ever made were:
- Leaving Vancouver to go to McGill
- Moving to NYC

I didn’t experience the ambition shade in Canada that Josh describes

But, in retrospect, I was 100% unknowingly in a relatively low-ambition bubble growing up in Vancouver. I didn’t see much entrepreneurship, or even corporate ladder-climbing. I didn’t see bold challenging of the status quo. I frankly just didn’t know what I didn’t know.

Putting myself in a position of being around hustle and grit and opportunity — seeing things I’d never even previously considered as plausible — shaped me more than I can describe

McGill and NYC happened to be those environments for me personally, but they can come in many forms

For me, McGill was my “the world is so much bigger and there are so many more jobs out there than you could imagine” moment

And NYC was working on the 51st floor in a tower on Park Ave and realizing that I was just as smart as all these guys. It’s where I started to take real agency over my career. I credit living in NYC entirely with me overcoming the Canadian inferiority complex that so many of us inherit for no good reason.

Now that I’m back in Vancouver, my hope is to be an example of that ambition and grit and challenging of the status quo for my kids and others. We need far more such examples in Canada.

TLDR: Get out of your comfort zone, see the world, ask the tough questions, chase any inkling of your curiosity, and tell your story.

[evan conrad @evanjconrad]

if you care about startups, have a balance sheet, and you would like to backstop this at a scale that is larger than what we can, we would like to work with you

if we don't fix this, we believe the industry will consolidate into a few large credit worthy players

dms open

[Silicon Data @silicondata]

In the current macro environment, raising capital to fund AI compute buildout has become increasingly challenging. This is true even for large established players—Meta acknowledged as much in its latest 10-Q amid rising commitments and cash-flow pressure—and far more so for the smaller so-called “neoclouds.”

This is occurring against a backdrop of rising compute rental rates across the entire term structure. Relentless demand for AI, visible in sharply higher prices, is precisely why so many are still racing to add capacity.

Institutional-grade, liquid compute futures based on transparent and trustworthy benchmarks are therefore both important and useful for the further development of AI in America. Neoclouds that execute well and deliver reliable performance can lock in future revenues, reduce risk, and secure cheaper financing as a result.

Silicon Data publishes daily market-implied forward rate curves for B200, H100, and A100. Once futures trading goes live, we will also have tradable futures curves. Current markets are displaying near-term backwardation (declining rates) and longer-term contango (rising rates).

[martin_casado @martin_casado]

Seriously one of the top systems experts, especially on syntax and model checking vulnerability detection and mitigation in the history of the field.

What are we even doing here … research considered harmful?

[b/acc, context platform engineer @AccBalanced]

Nikesh, this assumption is no longer true. I’ve been working on tokenomics of inference and RL efficiency for 2 years now. DeepSeek’s DeepInfer, and Nebius’ benchmarks of WEKA AMG (ahead of their successful cache token rollout) are objective proof points that:
1. Defender swarms are both economically sustainable, and
2. Low latency reactions at machine speeds are viable.

My X feed and Linkedin are full of supporting examples. Happy to elaborate with your tech team.