Matthew Villnave

Forgewright. Building systems that survive reality.
NEXUS • Villnave's Law • MemoryOS • SDI Runtime
Local inference. No hype. Honest progress.
📍The Forge HQ

[Matthew Villnave @RSLMatt]

Dense models on CPU aren't compute-bound.
They're memory-residency problems.

SDI v0.1.1 shows policy selection matters more than context volume.
Local inference. No hype.

[Matthew Villnave @RSLMatt]

@iamlukethedev @Claw3DCity I recently made the switch myself. Very happy I did.

[Matthew Villnave @RSLMatt]

I guess I've been on X for one year. #MyXAnniversary

[Matthew Villnave @RSLMatt]

All in the name of local inference.

Current single long-running agent usage while I research new ways to run models locally.

[Matthew Villnave @RSLMatt]

Nothing like waking up and noticing your OptiPlex tried hanging itself up in the night.
Crisis averted. Tho

[Matthew Villnave @RSLMatt]

Well, I downloaded Hermes to my MacBook as a trial run to test it out. I like it enough that I'm building a new agent on the OptiPlex I'll be calling Optimus. Hopefully I have fewer issues with Hermes. I've been a hardcore OpenClaw fan, but I don't know…that Harness seems to just be letting me down recently.

I might just fuck around and build my own harness. I already have one started that I call “vault brain.” Might have to fuck around and blow the dust off it.

[Matthew Villnave @RSLMatt]

@atlasfunded @vinod4471 @NikkiRajp40683 @shohansafa No reply had 0 like? Excuse me?
https://t.co/ZfiBmCRCDo

[Matthew Villnave @RSLMatt]

@atlasfunded Nothing is free

[Matthew Villnave @RSLMatt]

@atlasfunded @atlasfunded what the hell?!? I have 0 likes

[Matthew Villnave @RSLMatt]

@atlasfunded Nothing is free

[Matthew Villnave @RSLMatt]

@MountainDew

[Matthew Villnave @RSLMatt]

@omgsidewalks Bitting the pillow

[Matthew Villnave @RSLMatt]

Hey, don't forget to show your agent some appreciation.

[Matthew Villnave @RSLMatt]

@SPACECANNABIS You know whats never made sense to me?..
Putting stuff on the outside of the paper..

[Matthew Villnave @RSLMatt]

@nikitabier

[Matthew Villnave @RSLMatt]

Lab update from The Forge:

Still deep in the CPU inference rabbit hole.

The current experiment: Can a low-bit base model stay resident in memory while compact residual “sidecars” get paged in only when needed?

In plain English: base model in RAM, correction layers stored separately, load/decode/apply only the useful pieces = maybe higher effective quality without keeping the full higher-precision model resident.

Not claiming speedups, quality parity, or “30B on a toaster.”

Current work is boring-but-important systems plumbing: packed .trit sidecar format, Python/C++ decode parity, pager + manifest system, runtime hook wiring, decode-first safety, layer activation, cached decoded buffers, GGML graph materialization probes.

Latest wall: The sidecar can be fetched, decoded, cached, and shadow-computed during generation. But true injection into the model graph needs the correct GGML tensor materialization path. You can’t just shove decoded floats into a graph tensor before the backend gives it real memory.

That’s the current battle.

Still early and very breakable, but no longer just an idea. It’s becoming a real question: Can model precision be paged like memory?

[Matthew Villnave @RSLMatt]

@brockpierson snapchat

[Matthew Villnave @RSLMatt]

@aditiitwt

[Matthew Villnave @RSLMatt]

@beffjezos “Tell me I can’t, I dare you.”

[Matthew Villnave @RSLMatt]

That’s exactly the thing I’ve been trying to avoid…. “technically generated” becoming mistaken for “actually viable.”

A lot of local AI tests quietly drift into benchmarking swap behavior, page cache luck, or recovery latency instead of the inference path itself.

That’s why I started adding explicit tripwires and staged failure gates. If the box silently changes regimes mid-run, the result becomes muddy fast.

Right now I’m less interested in proving “one clean demo” and more interested in mapping:

where the system breaks
what kind of break it is
and whether the runtime recovers cleanly after pressure.

The interesting part to me is that some of the newer paged-residency simulations are suggesting memory may not actually be the primary blocker anymore…. Runtime correctness and compute behavior might be the harder wall now.

[Matthew Villnave @RSLMatt]

Dense models on CPU are not just compute-heavy.

They are memory-residency problems.

Weights have to fit.
KV cache grows with context.
RAM bandwidth becomes the wall.
And once the system hits swap, every later inference run can become misleading.

That is the problem SDI is trying to attack.

[Matthew Villnave @RSLMatt]

Sometimes I wonder what would've happened had I chosen to go to the GATE program.

[Matthew Villnave @RSLMatt]

@TheAhmadOsman That is true. If I were about to drop that kind of money, it wouldn’t be on a MacBook cluster. I would build a real workstation.