Today we introduce Heckler by @MercityAI
Heckler stress tests your AI agents with simulated users and shows you exactly where your agent breaks the rules you gave it.
We ran 3 agents (DeepSeek V4.1 Flash, GLM 5.3 and GPT-6 Luna) on Heckler and over 80% of chats broke a policy.
Weβre opening a research preview of Claude Duet, a new app where Claude works as a proactive agent.
It comes to you with what needs your attention, completes tasks you ask of it, keeps working when you step away, and checks with you before it acts on your behalf.
Constrained decoding already means one thing.
Keep generating, and mask any token the grammar rejects.
ScaleDown's CDMs are a different runtime.
Classify does not write the label. It scores the labels you supplied, softmax-normalizes them so they sum to 1, and returns top_label.
Use that when the schema is closed. Routing, tagging, fields you already named.
Do not treat top_label as abstention. The distribution always has a winner. Refusal is a threshold you set, or an "other" label you added. The model will not invent either.
The useful split is not how fast the loop is.
It is whether you still need a decoder.
You already gave the LLM the answer. So why is it writing it out from scratch?
Introducing a new AI model architecture: Constrained Decoding Models (CDMs)
LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.
CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.
Not just a decision model, but a family of 3 CDMs live today:
β’ Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
β’ Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
β’ Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.
Built for production:
β’ 1M tokens of multimodal (text + image) input, 31x larger than Jev
β’ 77.2% on DecisionBench vs Jev's 72%
β’ 200x faster, 467x cheaper than frontier LLM APIs
β’ $0.05/M input tokens. Output is free (yes, even for summarize)
Live on your favorite platforms today:
@OpenRouter@vercel_dev@LiteLLM@LangChain@TrustedRouter@itsArthurAI@useRouterPlus
Get 50M input tokens for free (for the first 50 hours) at link below π