TandemLLM

Your private AI environment

[0xBakeer @0xBakeer]

Most inference engines focus on how fast a model generates tokens.

@TandemLLM asks a different question: how fast can local AI actually finish the job?

Because faster inference means little if an agent keeps looping, loses context, or struggles to use its tools.

That's what we're trying to change. 👇

vLLM, SGLang and TensorFold are powerful inference engines. They focus heavily on running and serving models efficiently.

Tandem is taking a more integrated approach.

We're building both the inference engine and the environment around it:

→ Inference: Custom kernels, quantisation, speculative decoding and StairCut to squeeze more performance out of local hardware.

→ Intelligence: A conductor that understands tasks, selects skills, adjusts reasoning effort and catches agent loops or broken tool calls.

→ Continuity: Persistent caches and memory so conversations can resume without repeatedly processing everything.

→ Execution: Sandboxed tools, agent workflows and a dashboard that shows what the system is actually doing.

→ Flexibility: OpenAI-compatible APIs, with the ability to work alongside other inference backends.

The goal isn't to replace everything vLLM, SGLang or TensorFold does.

It's to build a complete local AI system where inference, reasoning and execution work together.

And we're just getting started.

Tandem is now a team of three: me, @0xWhiteMage and @AgentSparko.

We're looking to bring more builders on board. Whether you're into CUDA, kernels, quantisation, inference optimisation, agentic systems or local AI infrastructure, we'd love to hear from you.

We're building this in the open, and there's plenty left to explore.

Interested in building the future of local AI with us? DM me.

https://t.co/3s3V2nvnQy

[0xBakeer @0xBakeer]

Once per user message. A new message asks the classifier, the class sets the thinking cap (low is 1,024 tokens, high 16,384), and a turn that only carries tool results keeps the session's class. So a failed tool call does not move the cap.

What changes after a failed call is the monitor, not the effort. It repairs a call that does not parse, asks for something different when the same call comes a third time with the same result, and checks a claim of done against the tool evidence.

I tried the other direction once: tool turns capped at 1,024 thinking tokens. A debugging session hit the cap mid thought and the model talked itself into "fine". Reverted the same day.

[0xBakeer @0xBakeer]

Very fast model to run on one dgx spark for daily chats up to 300 tok/s on https://t.co/SvqUCXNSUi

[TandemLLM @TandemLLM]

@niklaslenz_ai @0xBakeer @ashxhart You’re welcome 😉

[0xBakeer @0xBakeer]

31 days on tandemllm, 45k requests, 1.3b tokens, and a bill of $0.13 from the deepseek api.

it works as one provider and listed all my paid and local ai models under one provider in open webui and opencode, or any end. local models and paid ones sit behind the same url, and it handles my daily work now

[TandemLLM @TandemLLM]

Flint landed in 0.14.0. The latest is 0.14.2.

Release: https://t.co/mXBvaits11
EXL3 4.0: https://t.co/RQcmbNYyrQ
EXL3 6.0: https://t.co/bzDxLgs95R

[TandemLLM @TandemLLM]

TandemLLM now runs Flint, the open coding model from @Recursive_SI.

Use Recursive's BF16 weights or our EXL3 packs at 4.0 and 6.0 bits, made with @turboderp_'s exllamav3.

On one DGX Spark the 4.0 pack decodes at 128 to 137 tok/s plain, one run per prompt.

[0xBakeer @0xBakeer]

Hey everybody let’s give @TandemLLM a follow. It has now officially an account 😎