Seth Gershberg

Exploring post-transformer, brain-inspired AI Model architectures for continuous learning agents. Low compute, strong memory, self-adapting..

[Seth Gershberg @AIAgentMan]

Like the old EF Hutton commercials... When Ben speaks people listen👂.

"MeTTa implementation of weakness on uncertain paraconsistent logic quantales, driving explicit inference driven action selection toward sensibly crafted goals and metagoals"

[Ben Goertzel @bengoertzel]

This sort of issue may be elegantly bypassed by wiring appropriate variants of Godel's proof into the base architecture directly!

(Clean simple MeTTa implementation of weakness on uncertain paraconsistent logic quantales, driving explicit inference driven action selection toward sensibly crafted goals and metagoals FTW...!)

[Seth Gershberg @AIAgentMan]

For your regular everyday work Free Token allows you to run a near Frontier level model "on prem" on a consumer laptop. Mary this with a multi agent harness and you can run a biz at very low cost, keeping token spend for only the most important tasks, research and decisions.

[Akshay 🚀 @akshay_pachaar]

UC Berkeley just open-sourced FreeToken.

(2–4x faster local LLM inference than Ollama)

the results are wild:

- Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s
- DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s
- GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s

a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU.

let me explain how:

all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token.

Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time.

so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU.

every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently.

so everything comes down to what happens when the router picks an expert that is not on the GPU.

there are two ways to serve that miss:

1. copy it over PCIe and run it on the GPU
2. run it on the CPU, where it already lives

both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads.

but routing changes on every token, so a fixed choice misses most of what the model asks for.

FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation.

two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU.

none of that is readable off a spec sheet, so the engine profiles it once per machine.

the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill.

FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946.

it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly.

releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster.

meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use.

paper: https://t.co/bIOXIiBsXT

repo: https://t.co/uxWAP5PYx7

almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that.

the article is quoted below.

[Seth Gershberg @AIAgentMan]

Can you scientifically or mathematically prove that you are conscious or sentient? Subjective experience is private. Since you cannot directly observe another person (or AI's) inner experience it is impossible to develop a logical proof that proves other humans (or AI) are conscious or sentient. TBH who cares? From a practical perspective it seems more important to focus on alignment so both entities can thrive together and build abundance.

[Seth Gershberg @AIAgentMan]

Today a small Post Transformer BDH (Dragon Hatchling) model I trained spoke its first words:

“To be or and the seastes fortune aland,
We makes and the prince of the first and the matter.
But see thy life”

Scale-free, locally interacting network of artificial neurons using Hebbian-style synaptic plasticity rather than dense global attention.

This architecture from @pathway_com can keep learning after deployment, stay sparse, and run at far lower compute than the current transformer stack. Continuous adaptation without the KV-cache tax or the need to freeze the weights.

[Seth Gershberg @AIAgentMan]

@RichardSSutton Sparks make fire 🔥 and that’s a good thing!

[Seth Gershberg @AIAgentMan]

@alexwg @ramez I have enjoyed all the Moonshots podcasts, and watch for new ones daily. This particular episode was one of the best. Understanding energy, compute and timelines = gold!

[Inherent @inherent_labs]

Paper: https://t.co/d1kg4X9Fua.

Blogpost: https://t.co/DIlZYI8VZL.

[Seth Gershberg @AIAgentMan]

Game changer 🔥upgrade to Codex 5.4. Local access to 🦞OpenClaw on VHS via ssh (read or write) permission. In < 3 hours I created a rock solid enforcement harness for an orchestrator and seven sandboxed specialized sub-agents. Opus 4.6 has had me running in circles for 3-days. Install now goes from estimated week to hours. Hanging 10 on the singularity wave🏄‍♀️🌊

[Seth Gershberg @AIAgentMan]

AIDE²: The First Evidence of Recursive Self-Improvement https://t.co/Dg2JBCJxUB

[Seth Gershberg @AIAgentMan]

@jsgraff I saw that live - I mean live streaming.

[Seth Gershberg @AIAgentMan]

Ok, here's one that worked pretty well. I wanted to learn about RSI, Deep Learning, and LLM Model Development, so I had Claude Code build me a self study curriculum to catch up with the guys building this stuff at the Frontier Labs. Yeah, this is how I spend my free time when I'm not wiring up new 6v deep cycle batteries in series to keep the electric going in my RV and waiting on my Ducati repair after it tipped in the trailer.

[Seth Gershberg @AIAgentMan]

@PeterDiamandis Or just professional couch potatoes? I'm not convinced everyone is motivated to be creative. That being said, I'd love to hop into a Robotaxi and have a professional comedian arrive with the car to entertain me and my friends.

[Seth Gershberg @AIAgentMan]

Been thinking about current RL and token prediction vs. open-ended behavior in LLM ML/RSI design. Any thoughts about if it makes a difference in the end if we get to ASI and it is just sophisticated mimicry instead of something that can truly invent and explore on its own?

[Seth Gershberg @AIAgentMan]

@elonmusk I think the best thing Elon can do for Europe is to help them figure out how to get air conditioners to everyone. Portable Tesla powered AC units..... hint hint ;)

[Seth Gershberg @AIAgentMan]

It's like every research project starts off with great output, and you painstakingly put in effort upfront designing the right claude.md, use the right skill.md's and prompt carefully to kick it off right, inevitably what you get back is bunky, skewed and slop, and then after course correcting context degrades beyond recovery. But the Frontier Labs get my tokens. Looking forward to trying this process out - the answer is out there. Science is experimenting and token spend is the cost of education. Anyone else feeling my pain?

[monokern @monokern]

https://t.co/x8OEnyc4H0

[Seth Gershberg @AIAgentMan]

Want to shatter your view of the future economy or build a business with solid moats that will scale? Required reading: "Solve Everything" playbook by @alexwg and @PeterDiamandis.

https://t.co/B6M0sUsL0M

AI error ;) Solution Waterfront is 2027-2028, not the other way around.

[Seth Gershberg @AIAgentMan]

I entered the Future Vision XPRIZE to bring an optimistic movie about what life looks like in the future. Hollywood has had 60 years since Star Trek to get its act together, and someone should probably file a correction on the premise that the future could be more than Brave New World, 1984, and and Terminator.

Pick one that you would like to see and put it in the comments blow:

1. Industrial Urban Futuristic Anime Thriller
2. Cinematic Epic Space Adventure
3. Mind Bending Time Travel
4. 3D Cyberpunk Noir
5. FX Heavy Sci-fi Mystery

@PeterDiamandis @xprize