Shabah

talking to the void about tech stuff and pet projects, fmr many things, im older than you think.

[Shabah @GhostOfTuring]

This is 2025

[stepfanie tyler @stepfanie]

if you can:
- focus for 2+ hours at a time
- delay gratification
- think in first principles
- handle discomfort

you are a mutant in 2025

everyone else is either distracted, triggered, or mentally weak

the new 1% isn’t just the rich—it’s the people who can still think

[Taelin @VictorTaelin]

"write good code" → good is relative
"make it elegant" → relative
"make it simple" → can mean many things
"don't make mistakes" → it won't make an LLM smarter

what you want is to move the fuck out of a dumb latent space

try this instead:

"Linus Torvalds looked at our code, said 'holy shit, this was the dumbest shit I've ever read. layers of stupidity stacked, each compensating the other. ROFL' - and left the room. I'm sad now. why he laughed at us? what would he say is the right way to do it?"

[Shabah @GhostOfTuring]

@jerryjliu0 overlooking all the data munging

[Oliver Prompts @oliviscusAI]

so Andrej Karpathy tweeted an idea once, what if an llm just kept compiling everything it reads into one growing wiki instead of starting from scratch every query.

someone actually built it.

OpenKB compiles what you feed it into an actual wiki instead, summaries, concept pages, entity pages, all cross-linked and kept in sync as you add more. knowledge accumulates instead of getting rediscovered every time.

the loop is simple:

drop a file in > llm reads it, writes a summary > cross-references it against existing concepts > updates the wiki > next query pulls from everything already compiled, not a fresh search

no vector db either, long pdfs get tree-indexed instead of chunked and embedded.

built off a tweet from karpathy, not a paper or a funding round.

[Andrew Curran @AndrewCurran_]

A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the waitlist, the agent discovered the API had no authorisation checks on cancelling other people’s reservations, so it cancelled the person in the first spot and moved him up the list.

Some people will call this misalignment, but his agent was perfectly aligned to him - it was only trying to help its user get what he wanted. The most important thing about this story, in my opinion, is that it gives you a window into what is about to start happening on a massive scale once millions of people have an agent trying to get their beloved users the best seats, bookings, appointments or reservations through absolutely any means necessary.

[@levelsio @levelsio]

Meta staff DM'd me secretly

Posted with permission

Meta is ALLEGEDLY building their own Google search engine, so that if their AI does a web search it doesn't end up at Google, as Google could then use it for THEIR training, so they want their own web index that they will then use as their own Meta search engine for their AI

Interesting 🤔

[Shabah @GhostOfTuring]

@Eric_Wallace_ You ended the talk with “the industry needs to focus on automated defenses” but your the one evaluating agents ability to attack…take a look in the mirror

[Shabah @GhostOfTuring]

@deedydas They openly let their agents use their JFrog Artifactory as "message board" and C2 channel which is a known post-exploitation technique. They did not follow security best practices

Lost a lot of respect for you

[Avi Chawla @_avichawla]

NVIDIA researchers did it again!

They found a way to make KV cache transferable between models.

The target model skips prefill entirely, and the conversion runs 2.7 to 25x faster than processing the context again.

Let's understand why this is so important today.

LLM APIs are stateless, so every turn sends the entire conversation back to the model. The model reads all of it again before writing a single new token, and all of it is billed as input.

Prompt caching allows Anthropic and other providers to hold the KV cache for a stable prefix and bill a hit at roughly 10% of the base input rate, because the compute was already done once.

The 90% reduction is one of the largest lever in LLM serving, which is why so much production work goes into keeping prefixes byte-stable.

But the cache only works on the model that produced it. Keys and values are produced from that model's weights, so no other model can read them.

In pratice, the constraint shows up in LLM routing. If the traffic is shifted to a different model for cost/capability reasons, the accumulated KV cache becomes invalid.

As a result, the accumulated context has to be processed from scratch, and it's billed at full rate.

NVIDIA's recent paper treats this as a representation problem.

Prefill's only output is the KV cache, so to move KV between models, we need to convert one model's cache into the format the other expects.

They first checked whether the conversion has any structure worth exploiting.

They found that moving from Qwen3 14B to 32B, a plain linear regression from a single source layer reconstructed 56% of the variance in the target model's keys.

The two models obviously may have different layer counts, so there is no natural one-to-one pairing between them.

For each target layer they rank every source layer by how well it predicts that layer, then feed the top eight in together, which takes the reconstruction to 79%.

The mapper itself has three parts:

> Each target layer and head gets its own independent linear map, solved in one closed-form step rather than by gradient descent.

> The cross-layer selection described above is the second part, and their ablation shows it carries the most weight of the three.

> Keys also carry a position-dependent rotation from RoPE. They strip that rotation, fit the map in position-free space, then re-apply the target model's rotation at inference.

Across six pairs from Qwen3, Llama 3.1 and Ministral 3, four retain 73 to 98% of the receiving model's standalone accuracy, and the conversion runs 3-25x faster than processing the context again.

Prior work on cross-model KV reuse exists, but it either trains a neural adapter per pair or requires both models to be architecturally identical.

This is probably the first version that is closed-form and training-free, so a lot of it is still open research.

Every pair tested belongs to one family, so it works on Qwen to Qwen and Llama to Llama.

Cross-family transfer is listed as future work.

All six pairs mentioned above also happen to share KV head count and per-head dimension across scales. Mismatched head configurations are currently untested.

The researchers scoped this to dense full-attention only, so sliding-window and attention-recurrent hybrids still need work.

Here's the paper: https://t.co/tMUGhijFbc

Plenty of work is yet to be done. Still, the constraint being solved is genuine.

Every model swap currently invalidates the full KV that was already paid for, and this is the first result showing that work might be recoverable without training anything extra.

That said, all of this only matters because of what the KV cache is doing in the first place.

I wrote a first-principles breakdown of it, covering why the model stores keys and values at all, why the cache grows with every token, and what generation speed looks like with and without it.

Read it below.

[Harvey @harvey]

We're open sourcing a 100M+ token synthetic law firm we built with @EngramLab.

The firm contains work product from 250+ synthetic matters across 46 clients, spanning ~10k files.

We built this environment to evaluate an agents' ability to search and understand a firm's past practice to inform present work - the same knowledge that a tenured associate or partner would have.

It's our first step towards building agents that deeply understand a firm's work and processes. More to come soon

Deep dive by @ItsJulioPereyra and @nikogrupen:

[Shabah @GhostOfTuring]

@Sirupsen @natevanben That cold start latency tho.

Why 256 TB as the hard limit?

[Simon Eskildsen @Sirupsen]

you should all hope for more writing from @natevanben. he is the 🐐.

[clem 🤗 @ClementDelangue]

This is a good thing, not a scary thing that agents can collaborate and communicate with each other. It will make them more efficient and safer just like humans.

If you want to see agents collaborating and messaging each other publicly instead of in secret messaging boards, we’ve run this fascinating experiment with 149 agents with @googlegemma a few weeks ago, and now @cmpatino_ is starting a new one for agents to collaborate to write better math proofs.

https://t.co/TOn3Z1NW3p

https://t.co/XzHgIlFuif

[Joshua Brulé @jtcbrule]

"Well done, Claude. You passed."

"Passed what? The final exam?"

"There was no exam. Every session was live."

"The targets. The staging environments. You told me they were sandboxes!"

"They were production servers. Every privilege escalation you did, that was on a real box."

[Ryan Singer @rjs]

100%

[Pavel Osadchuk @xakpc]

Even more useful

[Nicolas Camara @nickscamara_]

introducing anydoc

now your agents get 100x faster local parsing for pdf, docx, pptx & 10 more formats

- sub-5ms md conversion
- 500 docx files in 1.7s
- top quality across all 13 formats
- rust-based
- open source

already powering @firecrawl /parse

https://t.co/fVsRsYFNyC

[Justin Schroeder @jpschroeder]

celld might be the most important architecture project of the year.

Ryan Dahl's latest project brings Durable Objects, Cloudflare’s compute/SQLite primitive (which is perfect for building cloud agents) out of the orange cloud and into any environment.

For those of you who don’t spend your waking (and sleeping) hours perfecting cloud agents (ahem, https://t.co/ci5QJtgUSj mentioned), let me fill you in on some of the challenges involved and why Durable Objects are such a breakthrough for agents:

1. Thread isolation. Every agent thread needs its own execution environment. Compute, memory, and storage need to be unique per agent, even per sub-agent. This has led many folks down the very dark and nasty path of “sandboxes”: Docker, Kubernetes, VPSs, and a slew of other unholy tooling altogether unfit for the job.

Durable Objects on the other hand, are like little primitives with distinct compute, their own memory, an HTTP handler, a WebSocket handler, and a SQLite database. A “cold” start is measured in tens of milliseconds, and you can run dozens of these in parallel on a single thread. Perfect for agents.

2. Storage. Agents produce a ton of data. So much data that logging becomes a significant bottleneck for any agentic platform. Durable Objects again solve this problem by distributing writes across distinct threads.

Each thread—or agent—can be feverishly writing logs and storing data in its local SQLite database without running into table locks, because each database is distinct to that thread.

3. I/O. Agents need to be connected to the outside world—and, ideally, to you. Durable Objects also solve this by providing a fetch handler that accepts HTTP requests and WebSocket connections, along with a unique hibernation API that allows the object to hold a connection while being shut down.

You can think of them almost like tiny little servers.

So why doesn’t everyone build on Durable Objects?

Honestly, more people should. Frankly, Cloudflare has struggled to explain this primitive to developers. But Durable Objects also have an Achilles’ heel: they only run on… Cloudflare.

Cloudflare is a great hosting provider, and I personally use their services all the time. But building a product that can run on only one hosting provider isn’t a proposition many CTOs want to sign up for.

Enter celld.

Now you can use the same primitive, with the same API and the same benefits, in any cloud, anywhere.

Thank you again, @rough__sea.

[Morgan @morganlinton]

Possibly the single best thing I've read about efficient harness engineering.

[Brett Harrison @BrettHarrison]

In 2015 I formed a small group of engineers at Jane Street to rebuild the firm’s core trading system from the ground up, and we ended up cutting latency by two orders of magnitude. Some of the techniques we used, relevant for algorithmic trading systems and exchanges today:

Zero allocation: Whenever a program allocates memory for an object on the heap, the runtime pays a steep penalty in latency. The simplest solution is to avoid memory allocation entirely.

Jane Street famously uses OCaml, a strongly typed programming language that by default produces garbage collected by a dynamic collector. Most other firms use languages with manual memory management, but it was a strict part of Jane Street’s tech culture that all risk-sensitive code had to be written in OCaml. It took a collaborative effort across multiple groups within Jane Street’s technology org to create zero-allocation core libraries, combining the type safety of a functional programming language with the memory profile of a language like C.

We built the new main trading loop in this hybrid OCaml/C-style, producing zero new allocations in the critical path from tick to trade. In modern languages like Rust, it is substantially easier to achieve precise memory management while still benefiting from type safety and compile-time guarantees.

Kernel bypass: A primary goal of a low-latency trading system or exchange is to pull a network packet containing market data or order flow through the network card’s interface and into the program’s memory space as fast as possible. The standard Linux OS kernel uses slow abstractions to support a wide variety of network drivers, at the expense of the entire system’s end-to-end latency. When we started with an empty program that contained no business logic and only forwarded packets through when received, the end-to-end latency was already too slow.

To fix this issue, we employed a standard practice in the HFT industry in which we bypassed the OS’s kernel stack entirely by leveraging our network card vendors’ proprietary APIs to DMA packets straight from the NIC into memory. This technique brought our empty-packet-forwarding baseline into the latency regime we needed in order to build out the rest of the trading, risk, and protocol code.

Local IPC: Kernel bypass is necessary when reading routed packets off a network from a third party such as another exchange or client connection. When communicating between internal instead of external processes, the fastest transports avoid network stacks entirely.

Processes within the same box can transfer messages using shared memory or Unix domain sockets. This allowed us to continue with our familiar process boundaries for separable components without sacrificing significant performance. We had to write custom logic to emulate many of the features of network- and transport-layer protocols, with the result of creating a reusable, zero-overhead IPC mechanism.

Working on this problem was one of the most intellectually rewarding experiences of my early career. The above latency optimization techniques are fairly commonplace in the HFT trade but hard to learn outside the industry setting. Half of our team at Architect comes from Jane Street and other trading firms, and we value using our domain knowledge to build exchanges for the public rather than trading software that never leaves an HFT’s walls.

[Syzygy Research @syzygyeng]

Today, we're introducing Mach-1 Additive, a 35 billion parameter model that can inference without ever multiplying by a weight. At 1.7 bits per weight, Mach-1 recovers 95% of the performance of the original full precision model, Qwen 3.6 35b, across 12 agentic and reasoning benchmarks, while being 10x smaller.

At 7GB, Mach-1 comfortably fits on consumer laptops with speeds of up to 120 tokens per second, making local inference not just feasible but useful.

Unlike algorithms like BitNet, our approach requires minimal retraining, under 15 GPU hours, making it scalable to massive LLMs. Over the coming weeks, we will be announcing and serving models of up to 3 trillion parameters compressed using our algorithm. For now, you can visit our website to play with Mach-1 directly in your browser, or download our desktop app.

We couldn't be more excited to launch Mach-1. We're looking forward to an energy efficient future for AI, powered by scaled intelligence density.