tamas

ai engineering director, trailblazer, ducati rider, father and a good husband
¯\(ツ)/¯

[tamas @darvasch]

There’s absolutely nothing to worry about, as long as it’s fully unit, integration, regression, end-to-end, performance, load, stress, security, compatibility and chaos tested, with 100% deterministic CI/CD, exhaustive static analysis, mutation testing, zero flaky tests, full observability, reproducible builds, verified database migrations, backward-compatible APIs, canary deployment, automated rollback, and a completely green production-equivalent staging environment.

LGTM. Ship it.

[tamas @darvasch]

@Alibaba_Qwen @MiaAI_lab appreciate the work you guys are doing, kudos

[tamas @darvasch]

@yabaizettai @MiaAI_lab depends on your use-case

[tamas @darvasch]

@MiaAI_lab if you buy that, you are going to regret the 1TB NVMe SSD later. buy the bigger brother

[tamas @darvasch]

what happened to chatgpt 5.6 sol pro, it answers immediately with crazy speed :)

[tamas @darvasch]

@theo congratz, way to go

[tamas @darvasch]

That is what you say when your benchmark has two columns and your production system has never met a real user.

Price per task and speed matter, but so do correctness, reliability, safety, latency variance, observability, recoverability, and whether the “completed” task is actually correct.

A cheap, fast hallucination is not task completion. It is automated failure.

[tamas @darvasch]

Local 118B AI at 65.8 tok/s on a #DGX #Spark.
Yes, really.

[tamas @darvasch]

I just ran #Poolside’s #Laguna S2.1 NVFP4 locally on an @NVIDIAAI #DGX Spark with @vllm_project and @NousResearch Hermes Agent.

The surprising result:

DFlash can make it 3x faster.

Or make it slower than normal decoding.

Real measurements below 🧵

[tamas @darvasch]

My conclusion:

• General Hermes Agent use: run Laguna without DFlash
• Long, predictable code generation: use DFlash with 15 draft tokens
• Avoid blindly optimizing for peak tok/s
• Measure end-to-end performance on the actual workload

Local agents are becoming genuinely practical on a single DGX Spark.

[tamas @darvasch]

I just ran #Poolside’s #Laguna S2.1 NVFP4 locally on an @NVIDIAAI #DGX Spark with @vllm_project and @NousResearch Hermes Agent.

The surprising result:

DFlash can make it 3x faster.

Or make it slower than normal decoding.

Real measurements below 🧵

[tamas @darvasch]

But on less predictable reasoning and agent outputs:

• Acceptance fell to 2% to 8%
• Generation dropped to 10 to 18 tok/s
• Sometimes slower than the stable 19.2 tok/s baseline

The draft model was generating quickly, but most proposed tokens were discarded.

Speculation is not automatically acceleration.

[tamas @darvasch]

@mark_k true

[tamas @darvasch]

@kimmonismus yes, but...

[tamas @darvasch]

@maxedapps besides that ...