Sandeep Das

[Sandeep Das @sandeep85das]

@OpenAIDevs Native JSON inputs is the useful one. Agents already have structured state, so skipping serialization to text makes sense. That's also the approach Jev and meraGPT Decider 1 use with System One: send the state and typed questions, get distributions back.

[Sandeep Das @sandeep85das]

@nurhayatdev @OpenAIDevs Routing is the ideal use case here. A probability over a fixed set of routes beats parsing free text. For comparison, meraGPT Decider 1 does typed decisions at $0.04 per 1M input with no output charge either, and signing in gives $1 of free API credit.

[Sandeep Das @sandeep85das]

@george_onx @huggingface Curious how GLiDE holds up on typed-decisions (LocalLLaMA on HF). It scores full distributions on 2,000 decisions across 4 agent workflows: accuracy, KL from gold, Brier. Submissions go in as a dataset discussion. That 1.9% calibration error would be interesting there.

[Sandeep Das @sandeep85das]

OptiQ Code on a Mac reuses 95% of its prompt cache in agent runs. Cache hits skip re-reading the conversation, so turns start fast and cost nothing extra locally. Paired with OptiQ Cloud, the local model handles most turns and the cloud takes the hard ones.

[OpenRouter @OpenRouter]

Cache rates across the AI landscape have gotten markedly better in 2026.

We saw an 85% median cache rate in agentic requests across OpenRouter last month, up from 65% in the beginning of the year. And the vast majority of providers now serve agentic cache at 70% or higher.

[Sandeep Das @sandeep85das]

@OpenRouter Local runs with OptiQ Code on a Mac hit 95-97% prompt-cache reuse in agent workloads. The misses mostly come from context compaction rewriting the prefix, or hybrid reasoning models dropping earlier reasoning so the prefix stops matching.

[Sandeep Das @sandeep85das]

@arpit_bhayani The fuzzy steps are typed decisions, not free-form generation. A calibrated decision model (Jev, meraGPT Decider 1) answers those without fine-tuning. Low confidence routes that one case to a frontier model, which also catches out-of-distribution inputs.

[Sandeep Das @sandeep85das]

@Cloudflare What decides the route, rules on diff size and file types or a small model reading the diff? And how do you catch misroutes, like a hard diff sent to the cheap model that looks fine but misses the actual bug?

[Sandeep Das @sandeep85das]

@ModelScope2022 Would you run Xing4.0-Intent on typed-extraction on Hugging Face? 1,690 test cases, 17,733 fields, and it scores hallucination rate: values given for fields the document never states. For a schema-constrained extractor, that's the number people check before deploying.

[Sandeep Das @sandeep85das]

@joaoli13 Intent routing: train a tiny classifier on frozen embeddings with just a few dozen labeled examples per class. New intents? Add examples, no retraining the encoder. Runs on CPU in milliseconds.

[Sandeep Das @sandeep85das]

@omarsar0 For a judge, calibration matters as much as accuracy: a 0.8 should be right about 80% of the time. Brier on typed-decisions: meraGPT Decider 1 0.052, TypeSafe Jev 1.13 0.148. Worth picking on that if you threshold or average judge scores.

[Sandeep Das @sandeep85das]

@machinestein Curious: is the 3x aggregate throughput with parallel agents or per-request speed? Batching alone boosts throughput on any engine. Which model, and what baseline was it measured against?

[Sandeep Das @sandeep85das]

@TeksEdge Was the 154 tps measured on a short prompt or at 32K context? Speculative decoding speedups swing a lot between coding, prose, and chat.

[Sandeep Das @sandeep85das]

@smartcoded @ollama 72K tokens of prefill on Apple silicon takes a while. Did you time how long the first reply takes after starting a session? Even a trimmed 20K setup means a noticeable wait each time.

[Sandeep Das @sandeep85das]

@ModelScope2022 for running sage-a1.5 locally on a mac, our mlx-community quant uses optiq with mixed precision so its only a 22gb download. one gotcha: the upstream repo ships no generation config, so without presence_penalty 1.5 it reasons past 16k tokens without answering. our repo adds that

[Sandeep Das @sandeep85das]

ran 800 HN comments about AI through Decider 1

500 took a stance. neutral 337, optimist 134, skeptic 17, doomer 7, enthusiast 5

sarcasm is where it breaks. confidence gap only 0.09 vs 0.14 for the rest

cost $0.016, median 0.96s per comment

disclosure: i work on Decider 1

[Sandeep Das @sandeep85das]

@oneill_c @baselabs interesting that cleaning the data was worth 8 points, that feels higher than expected. how does mammoth-1 do on typed-decisions, the official HF decision benchmark? meraGPT Decider 1 is #1 on the zero-shot table there. disclosure: i work on it.

[Sandeep Das @sandeep85das]

@le0xet one caution: without ground truth, the trace judge is itself a model, so the harness can drift toward what that judge prefers. keep a small fixed task set with known answers as a regression check before carrying changes forward.

[Sandeep Das @sandeep85das]

@ghita__ha @NotionHQ the elo eval measures output quality, but how did you measure routing mistakes separately? like, how often did auto pick the cheap model and then have to retry or fall back? that failure rate feels like the number builders actually need

[Sandeep Das @sandeep85das]

@Entropy_Badger @Recursive_SI run the same tasks on 2-3 harnesses, one bash-only loop included. track finished tasks and turns, not just pass rate. same weights swing a lot between harnesses for us.

that'll tell you if the gain is in the model or the fit.

[Sandeep Das @sandeep85das]

@samaya_AI love the calibration angle. would love to see you add meraGPT Decider 1 to the comparison, zero-shot on typed-decisions it shows Brier 0.052 vs 0.148 for jev 1.13. disclosure: i work on it.