Oliver Frankfurth

AI & Tech Entrepreneur. Building self-hosted AI systems for small teams. Founder abmelden-de · Fundsback-org. No AI slop.

[Oliver Frankfurth @frankfurth]

What you get: sovereign AI. Control over data, models, iteration. The 'just works' story doesn't exist locally.
So: do you run the iteration yourself or hand it to a provider?

[Oliver Frankfurth @frankfurth]

Who's running local AI in production — not pilots, hobbies?
10 months in. 1 DGX Spark (OCR + customer data), then a Ryzen 128 GB (content), now 2 more Sparks (coding + agents).

[Oliver Frankfurth @frankfurth]

Cost: power + 2 small subscriptions for edge cases.
ChatGPT Enterprise for a 10-person team would be ~$650/mo. And the data would sit in Azure.

[Oliver Frankfurth @frankfurth]

Next step: see what's left to extract at this low-concurrency profile (2-3 streams) — most optimizations are designed for high-throughput production, not internal setups with thin load.
Anyone running 35B-MoE at low concurrency with tips?

[Oliver Frankfurth @frankfurth]

Spent today tinkering with model optimization on our DGX Spark setup. Others reported solid results with specialized Qwen3.6-35B-A3B variants — wanted to see how much of that actually shows up under my workload. 🧵

[Oliver Frankfurth @frankfurth]

We run 2-3 concurrent requests, no 128-stream production load. The boost still shows up in daily use.
Lesson: „defaulting conservative" can kill the very performance path you're trying to use.

[Oliver Frankfurth @frankfurth]

Two machines on a desk. Cost less than one year of ChatGPT Enterprise for a 10-person team.

Running Nemotron 120B (MoE, 12B active params) for reasoning and agent orchestration. Qwen 3.6 35B (MoE, 3B active) at 43 tok/s for fast tasks.

Both on NVIDIA DGX Spark. 128GB unified memory. FP8 KV cache. 128K context. 8 concurrent requests. Requests get prioritized — urgent first, batch jobs when there's room.

Claude? We use it too. But only when it matters: coding, architecture, complex reasoning. 90% of daily workflows run locally. No subscription, no API costs.

The key: we control what goes out. Customer data, financials, strategy — stays local. Always. When we use frontier models, we decide what enters the context. Nothing leaves the office by accident.

Can local models compete with GPT-5? No. They don't need to. Most business tasks need reliability, not benchmarks.

[Oliver Frankfurth @frankfurth]

I have a problem with AI content.

Not with the technology. With what most people do with it. Generic text that sounds like nothing. Interchangeable. Empty.

We serve 40,000+ clients across three websites. Small team, lots of content. Of course we use AI. But with one rule: every text gets reviewed, edited, and approved by a human before it goes live.

AI delivers the draft. Quality comes from us.

"More content" is not the goal. Better content is.

[Oliver Frankfurth @frankfurth]

@CryptoDadsNFT @JungleFreaksNFT @mp_software @YFrankfurth