Vikas Goenka

Ringarc ( — AI for Small Businesses & MSMEs | Fractional CTO & Technical Advisor | Ex-HFT CTO

[Vikas Goenka @vikasg_rac]

A price on one endpoint tripled and came back down inside twelve hours.

DeepSeek announced an API pricing change on August 13, effective the 16th. Peak and off-peak rates, with increases from 50% to over 1,100% depending on the model and token type. That part was public and widely covered.

What I could not find announced anywhere is when each provider serving those models would follow, or by how much.

Since August 20 I have been running a small monitor for my own open-model testing. It sweeps OpenRouter endpoints twice a day, 45 when I started and 165 now, and saves the listed price, latency and error rate to disk. Then it compares each day against the last.

Eight days after that took effect, my sweeps had logged eight price moves across five providers, six up and two down. GMICloud put deepseek-v4-flash up 90%. StreamLake put glm-5.2 up 106%. Four more worth naming:

- Baidu took deepseek-v4-flash from $0.049 to $0.14 per million input tokens, and $0.098 to $0.28 on output. That is +186%, landing exactly on DeepSeek's old flat list price. Twelve hours later the same endpoint was listing $0.0546 and $0.1092. Net move so far, about +11%.
- StreamLake moved the same model from $0.0489 to $0.0574, then settled at $0.0560. About +14.5%.
- Mancer raised gpt-oss-120b input pricing by 6%, from $0.080 to $0.085, on the 21st. Flat since.
- Darkbloom cut qwen3.6-35b-a3b on the 23rd, 12.5% off input and 6.7% off output. Flat since.

I do not know whether Baidu's $0.14 was a brief reprice, a staged rollout, or a bad listing. I only know it was there at 09:45 UTC and gone by 21:45, because both sweeps are on disk. A once-a-day check would have recorded one number or the other and called it the price.

It is not only price. The deepseek-v4-flash endpoint Mancer added on the 23rd got slower the next day, from a 4.8s median to 6.4s across 82 requests. That is what my sample saw, not a claim about the endpoint's whole day.

DeepSeek's announcement covered DeepSeek's own API. These four are third-party hosts running the model on their own hardware and setting their own endpoint prices, and I did not see any of them announce a change to those. Such announcements may exist somewhere I do not read.

If you use an aggregator with auto-routing, your per-token cost might have moved this week and nothing in your repo changed.

Quality changes happen. Costs change too. Almost nobody keeps the history that shows when a change actually reaches the endpoint you are calling. I am working on making that tracking boring and automatic.

[Vikas Goenka @vikasg_rac]

Sometimes you need to challenge your model. I did it this week on running Qwen3.8-27B and I am glad I did though the model was not entirely wrong.

For anyone who doesn't run models locally, open-weight models contain billions weights and this need to fit in your GPU memory. Quantisation makes those numbers smaller. A model can go from the original 16-bit weights to 8-bit, 4-bit, or even 3-bit. Each step cuts memory use and usually improves speed, but it can also reduce quality.

Where the model sits matters just as much. A model runs quickly when all its weights fit inside the graphics card's VRAM. If part of it lands in normal system RAM, generation has to use the slower memory bus. Performance can fall sharply.

I'd overlooked that part. For a month, I considered Qwen3.8-27B unusable on my 16 GB card because it managed only 6.75 tokens a second. The model was already quantised to 4-bit, but a 27B model at 4-bit is about 18 GB, so some of it spilled into system RAM.

Then someone online pointed out the fix. Move down to 3-bit so all the weights fit in VRAM, enable flash attention, and reduce the conversation memory by shrinking the KV cache.

My assistant agreed that this would improve speed, but warned that 3-bit would hurt quality. We had measured that cost before, so it told me to test instead of guessing. That's what we did.

The 3-bit "dynamic" quant keeps sensitive layers at higher precision and takes 13.4 GB. With the full model in VRAM, flash attention enabled, and the KV cache at 8-bit, performance went from 6.75 to 52 tokens a second. That's a 7.7x improvement from configuration alone, with no hardware upgrade.

There were tradeoffs. I had to limit context to 32k tokens. The model also scored 0.49 on our hardest evaluation, compared with 0.56 for the 35B mixture-of-experts model we use every day. The evaluation has 22 agentic coding tasks in a real three-database app. A mixture-of-experts model activates only a few "experts" for each word, which lets a large model run quickly on smaller cards.

The two models performed equally on localised bug fixes. The 3-bit 27B was worse on work spread across several files. But it was also the most honest local model when things went wrong. It made one false "done" claim across 13 failures, while the MoE made four.

There's a broader point here. My read is that dense models with 27 to 30 billion parameters, which are now appearing every few weeks, are sized around unified-memory Macs. Those machines can comfortably hold a 30B model at 4-bit or even 8-bit. A 16 GB gaming card needs 3-bit to make the same models fit. The other option is a MoE with a small number of active parameters.

Before evaluating a model, the machine needs the best configuration for that model class. I did it in the opposite order.

One question remains. Does the difference between 0.49 and 0.56 come from the models themselves, or from the 3-bit tax? The 4-bit 27B was too slow to run the evaluation usefully, so I still can't separate those factors.

Please comment what has been your experience running Qwen 3.8-27B locally.

[Vikas Goenka @vikasg_rac]

Blog post (how it works and the design details): https://t.co/COLEqXldn3
2-minute demo video: https://t.co/iRFY1tBGgw (Sorry in advance about the AI voice. I'll do it in my voice and reshare)
Code (MIT): https://t.co/rn1HhPqTAH

[Vikas Goenka @vikasg_rac]

Most of my work happens in Claude Code, often in sessions that run for hours. After a while, finding an earlier response becomes difficult. And continuous scrolling up and down to see those messages can be frustrating.

Most session output is meant for the machine. It includes tool calls, command output, and file diffs. My prompts and Claude's replies make up only a small part, but the terminal puts everything into one continuous stream. Even markdown appears as literal hashes. Web viewers can format transcripts well, but I didn't want to add a browser and server when I was already working in the terminal. I just wanted another pane.

That's why Claude Code and I built claude-reader. It's a tiny terminal app that follows the transcript Claude Code already saves in ~/.claude/projects/. It filters the transcript down to Claude's text and my prompts, then displays them
as markdown in a sidebar and reading pane. It refreshes twice a second.

There are no hooks, plugins, or server. The app only reads a file. It works in any terminal, with any split, and over ssh. Changes to Claude Code's UI won't break it.

Building for the pane beside Claude Code shaped a few choices I hadn't expected. Mouse support is disabled by default, and the header has no clock but has timestamp of last message. Mouse tracking and once-a-second repaints both stop drag-selection from working in the neighboring Claude pane.

I had a different model review it before release. That review caught four actual crashers. Claude Code fixed them, added tests, and released 0.2.0.

I have been using it for two days, it is working beautifully and I am finding it really useful.

Install: pipx install claude-reader (or uvx claude-reader). This is an independent project and isn't affiliated with Anthropic.

The blog link, 2-minute YouTube video, and repo are in the first comment. Please give feedback if you use it.

[Vikas Goenka @vikasg_rac]

After my RAG article, someone asked: "What's the difference between CAG and long context? To me CAG is cheap long context with no drawbacks."

Great question, because it's almost right, and the "almost" is where projects go wrong.

Long context has two problems, not one. It's expensive: you resend and reprocess your whole corpus with every question. And it's low quality: models get measurably worse at reading as the input grows (Chroma tested 18 frontier models, every one degraded, long before the window was full).

 CAG fixes only the cost part. You process the corpus once, save the model's working state, and answer every future question from it. But at answer time the model is still staring at the same huge pile of text. It reads it just as badly as before. Cheaper, yes. Better, no.

So CAG shines when the corpus is small and static, a product manual, a policy handbook. There, quality was never the problem, and cost was the only objection left. Once the corpus is big or changes often, the quality problem takes over, and making a badly-read context cheaper doesn't help. One edited document also throws the saved state away, and you pay full processing
 again.

One line to keep: long context's problems are cost and quality. CAG fixes cost. If your corpus is small and static, that's enough. Otherwise it isn't.

Part 1 of the RAG series (all five debates) is https://t.co/q2mYsH0fNi

#RAG #AIEngineering #LLM

[Vikas Goenka @vikasg_rac]

Here is the full article: https://t.co/cGTyzMyrtz

[Vikas Goenka @vikasg_rac]

Everyone tells you basic RAG is dumb. It is not!

Or put another way: your boring RAG pipeline is probably fine.

I read ten viral "RAG in production" posts side by side last week. Two things stood out. First, every single one presents "the problem is retrieval, not the LLM" as a contrarian insight. When every post makes the same contrarian claim, that claim is the consensus.

Second, they directly contradict each other. One lists Graph RAG as a top production mistake, three others sell it as the fix. One says start with dumb fixed-size chunks and measure, two others call that mistake number one. And my favorite: a whole article on how Google's new "Open Knowledge Format" is replacing the vector database. It is a spec for a folder of Markdown files. It replaces nothing.

So the average engineer reads all this and comes out worse than they went in: loaded with new terms, buried in techniques, and still unsure what to build.

Here is what I think is actually going on. There is no RAG debate. There are five separate debates that all got tagged #RAG, each with different evidence from a different domain, and each with a different answer. Grep vs vectors. RAG vs long context. GraphRAG. CAG. Memory.

Pull them apart and a single question sits underneath all five. Answer it for your corpus and your queries, and most of the noise stops applying to you.

Part 1 of the series is up on the Ringarc Labs blog. It also covers what the narrative oversell (chunking), what they skip entirely (the part where enterprise RAG actually bleeds), and I admit upfront which of my own numbers is made up.

 Link in the first comment.

#RAG #AIEngineering #LLM

[Vikas Goenka @vikasg_rac]

The tool that can generate slick launch videos keeps talking me out of making them.

I've been building and launching small projects with Claude Code, and I noticed a pattern. Every time I ask for something flashy, an animated demo, punchy marketing copy, it nudges me the other way. Use a real screen recording. Write it the way you'd say it. Show the actual terminal, mistakes and all.

Its argument, when I pushed back: polish has become cheap. Anyone can generate a beautiful video in an afternoon now. So beautiful no longer signals effort or quality, it signals nothing. What people actually respond to is proof that a real person built a real thing. A rough capture of it actually working. A sentence only you would write.

I didn't fully buy it at first, because the polished versions honestly look better. But every time I've posted the plain version instead, it's done better.

Bit ironic. The best marketing advice I've gotten this year came from an AI telling me to sound less like one.

[Vikas Goenka @vikasg_rac]

Everyone's giving Claude a brain. I just wanted to stop repeating myself.

For months my feed was full of "give Claude a memory" projects. I scrolled past every one of them.

Meanwhile, several times a week, I was hitting the same wall: re-explaining my own projects to Claude Code. Same context, again and again, across many projects and two machines. Then I found out Claude Code deletes your session transcripts after 30 days, on my machine, the oldest surviving one was 29 days old.

Those two things collided into a build. It's called project-brain, and I open-sourced it today: every Claude Code session, in every project, automatically captured into one plain-Markdown brain, archived, stripped of secrets, distilled into notes, compiled into a ~20-line index Claude reads at the start of every session.

The core idea is layers: full transcripts at the bottom (grep-able, never injected), distilled notes above, living project pages above that, a tiny index on top. Cheap questions get cheap answers; exact details are always one grep away.

What it's built for, in one moment: I asked which local model to use for a course project, and Claude answered with my own benchmark results from a different project, work I'd already half-forgotten.

Funny ending: when I finally read all those posts I'd scrolled past, our design matched what Karpathy calls the LLM-wiki pattern almost exactly. The wave was right. I just needed my own pain to take it seriously.

Full story, the research, the design philosophy, and what the redactor found in my own transcripts, on the Ringarc Labs blog: https://t.co/1YEwFicHjL

https://t.co/DSj6Dy8cwf It's v0.1, try it, break it, tell me what's wrong with it.

hashtag#ClaudeCode hashtag#OpenSource hashtag#AI hashtag#DeveloperTools

[Vikas Goenka @vikasg_rac]

Picture this. A customer messages your business on WhatsApp at 9:41 pm: "Need 200 pieces before the 15th. What's your price?"

By 9:42 they have a warm, accurate reply with your catalogue and prices. The enquiry is logged in your sales tracker, a follow-up is scheduled, your Salesperson is alerted. You read the whole story the next morning over chai — hottest leads on top.

No call centre. No night shift. No new software. This is what AI already makes possible for small businesses — inside the tools you use every day: Excel, Word, Canva, Tally, and above all, WhatsApp.

Many of you know me here for benchmark and engineering posts. This is the other thing I've been building: Ringarc, where I embed AI into Indian small businesses and measure the result in hours saved.

I've written a tour of what's possible right now. No jargon, no hype, and one non-negotiable: nothing leaves your company without human review. You stay the boss. AI is just a very, very capable genie.

 Read it here: https://t.co/zZ3eaIN6lz

If you run a business — or know someone who does — I'd genuinely like to hear: which of these would save you the most hours?

[Vikas Goenka @vikasg_rac]

Alibaba dropped Qwen3.8-Max yesterday, so I did what I always do with a new model now: ran it through my personal benchmark the same day and put it head-to-head against the current start of open models, Kimi K3.

Quick background: I keep a private benchmark of ~163 tasks based on realistic everyday work, coding, repo-level bug fixes, document extraction, tricky reasoning, OCR, awkward judgment calls. Frontier models (Opus, Sonnet, GPT, Gemini) are frozen in as reference anchors, and every new open model gets measured against them. Not a leaderboard, just one person trying to answer "everyone says X is frontier-level, is it actually?"

So, is it? Honestly... yes. First model I've tested that lands a hair above my frontier reference line. Two things genuinely impressed me:

1. It didn't fall for my hallucination bait tasks, fake movie titles, questions with false premises. Every previous open model I tested invented confident nonsense on at least some of these. Qwen just said "that doesn't exist."

2. It held up on full-repo coding tasks, where most open models (including Kimi) drop off noticeably compared to small isolated coding problems.

What's lacking? It talks too much. Ask it to explain UPI to your uncle in plain words and you get 400 words full of unexplained acronyms. Kimi is clearly better at these everyday, human-register answers.

And on my hardest test category, messy ethical judgment calls with no clean answer, Kimi got a perfect score on every single one, which no other model I've tested has done. To be fair, Qwen wasn't bad there either: it engaged every dilemma head-on and scored right around what the frontier models do.

The cost surprised me most. Running my entire suite: Qwen3.8-Max was roughly 3x cheaper per task than Kimi K3, generated ~40% fewer tokens, and was more than twice as fast. Frontier-level answers at a third of the cost is quietly the biggest story here.

Same tier, different personalities: Kimi for talking to people, Qwen for doing the work.

@Qwen team, the promised open weights are the part I'm actually waiting for. My local test rig is a single 16GB GPU with 128GB of system RAM, so please don't forget the 27B when you ship next week. The day the weights drop, I'll run it locally and put it through the full suite.

[Vikas Goenka @vikasg_rac]

Cloudflare’s Meerkat: Moving Beyond Leader‑Based Consensus for Global Control Planes

Cloudflare’s Meerkat is a new internal control‑plane service that uses the QuePaxa consensus algorithm to get strong, globally consistent coordination without a single leader, trading extra network round‑trips for better availability on Cloudflare’s worldwide network.

We’ve lived with leader‑based consensus (Raft, Paxos) for a long time and know about the operational pain of getting the right "Timeout" and regional hiccups stalling the entire cluster's write path.

Cloudflare’s Meerkat experiment is interesting because it actually changes the model, not just the implementation details. Meerkat is a global consensus service built on QuePaxa, a 2023 algorithm designed for asynchronous networks, no “eventual synchrony” crutch, and no dependence on timeouts for progress. All replicas can accept writes. A leader can exist to optimize, but it’s not a single point of failure the way it is in Raft. It exposes a consensus log with a strong invariant: once a slot is decided, every replica agrees on that value. That’s the basis for linearizable control‑plane operations.

This goes beyond Dynamo/Cassandra‑style leaderless replication. Those systems accept conflicts and push resolution into application logic; Meerkat is explicitly aiming for strong global consistency for config, leases, and other control‑plane state.

Example that makes this tangible:

You’re rolling out a new payment flow via a feature flag across India, Europe, and the US. The flip becomes one event in Meerkat’s global log. A majority of replicas agree on that event and its position in the sequence before it’s applied.
Every region replays the same sequence, so once the slot is decided, no data center can “disagree” on whether the flag is ON or OFF.

It’s still experimental, and it’s not a general‑purpose database. But if Meerkat delivers on its design, it’s a meaningful step toward making strong, global coordination less dependent on “one leader, tuned timeouts, and a prayer that the WAN behaves.”

https://t.co/12dIfWgg6g

#systemdesign

[Vikas Goenka @vikasg_rac]

@Salem_cygman https://t.co/5hhjANPce2

[Vikas Goenka @vikasg_rac]

A tiny OCR model that finally understands long, messy documents

One thing that really caught my eye this week: Baidu’s new Unlimited OCR is a long‑document OCR model that’s small enough to run locally, yet smart enough to read 40+ page PDFs in one go.

Instead of restarting on every page like most OCR tools, this model keeps looking at the whole document while using very little extra memory. So you don’t have to slice PDFs into chunks or worry about losing context. It can understand tables, layouts and long contracts or reports more naturally, even when you run it on your own machine.

For anyone exploring AI + productivity, this feels like a quiet but important shift: OCR is finally catching up with how we actually work with documents, long, messy, and very much more than “just one page at a time.”

[Vikas Goenka @vikasg_rac]

I spent a few evenings benchmarking three ways to run open-source LLMs on my desktop (RTX 5070 Ti, 16GB VRAM): Ollama, llama.cpp, and vLLM. Here are the results.

I started with two models picked to straddle the 16GB line: gpt-oss-20b (~13GB, fits) and Qwen3-Coder-30B (~18GB, doesn't fit). Partway through I realised the lineup was missing a third case, a model that fits with room to spare, so I added Gemma 4 12B (~8GB at 4-bit) later. Glad I did, because it changed the conclusion.

Round 1, model fits snugly (gpt-oss-20b): llama.cpp 226 tok/s, Ollama 166, vLLM 33. Yes, 33. vLLM couldn't load the model next to my desktop session, three out-of-memory failures, and its fallback streams part of the weights over PCIe on every single token. The bus became the speed limit.

Round 2, model too big (Qwen3-30B): llama.cpp again, 71 tok/s. It has a flag (--n-cpu-moe) that pushes only the "expert" layers of a mixture-of-experts model into system RAM and computes them on the CPU. Since each token only wakes a few experts, the CPU barely works. An 18GB model doing 71 tok/s on a 16GB card is amazing.

Round 3, model fits comfortably (Gemma 4 12B), and here the story flipped. Single user: llama.cpp 91, Ollama 87, vLLM 81. But with 8 clients at once: vLLM 419 tok/s total, llama.cpp 232, Ollama 78. The same burst of 8 requests took vLLM 4.9 seconds, llama.cpp 8.8, Ollama 26.3. Give vLLM resident weights and parallel traffic and you can see exactly why datacenters run it.

The Ollama result deserves its own line: 8 clients got the same total throughput as 1, in all three rounds. Out of the box it serves requests one at a time. It's fixable with one env var (OLLAMA_NUM_PARALLEL), almost nobody sets it I think.

So my takeaway isn't "which engine is best for local inference." It's: does your model fit, and how many people are hitting it?

- Fits, one user → llama.cpp
- Fits, many users or batch jobs → vLLM
- Too big and it's a MoE → llama.cpp's expert offload, nothing else is close
- Want it running in five minutes → Ollama, but set that env var

[Vikas Goenka @vikasg_rac]

How does local Inference Work?

I've been benchmarking inference engines on my home desktop and wanted to post the results. Then I realized the results only make sense if you know how local inference actually works. So, first things first.

Running an LLM locally is not a compute problem. It's a plumbing problem.

By plumbing I mean the pipes inside your computer that move data around. Chips don't compute on data where it sits. Data has to travel from memory into the processor's compute units first, and every pipe it travels through has a maximum flow rate. Your GPU's onboard memory (VRAM) is the widest pipe in the box: it feeds the GPU at roughly 1,000 GB/s on a modern card.
Regular system RAM, the sticks feeding your CPU, moves maybe 60-100 GB/s. And the PCIe connector between the GPU and the rest of the system is narrower still, around 30-60 GB/s. Wide pipe, medium pipe, straw. Remember that ordering.

Now the workload. To generate one token, a model reads basically all of its weights once. A 20B model quantized to 4-bit is ~13GB, so every token means pulling 13GB through a pipe into some processor. Want 50 tokens/sec? That's 650 GB/s of sustained reading. The processors themselves are rarely the limit. They're mostly waiting to be fed.

So the whole game is: which pipe are your weights sitting behind?

Happy path: the model fits in VRAM. Weights get copied there once, at load time. From then on, every token is the GPU reading its own onboard memory through that 1,000 GB/s pipe. This is why a 16GB card runs a 13GB model at 200+ tokens/sec. Nothing ever crosses a slow pipe.

But what if the model doesn't fit in VRAM? Say it's 18GB. The naive fix is to park the overflow in system RAM and copy whatever the GPU needs across PCIe, every single token. Through the straw.

The smarter fix is where mixture-of-experts models shine. A MoE model is big, but per token, it only activates a few "experts," a small fraction of the weights, maybe 3B out of 30B.
Catch: it picks different experts each token, and you can't predict which. So all the weights must stay loaded and reachable somewhere. You just don't need them all through a fast pipe on any given token.

That enables a neat trick: keep the parts used every token (attention layers, shared weights) in VRAM, leave the huge pool of sometimes-used experts in system RAM, and when a token needs one of the RAM-resident experts, don't move it. Let the CPU compute that piece right where the data lives. Only tiny intermediate results cross PCIe, kilobytes instead of gigabytes. Each processor works on what's local to it. That's how my 16GB card runs an 18GB model at 70 tokens/sec instead of 5.

Same math either way. The difference is whether your data crosses a straw or stays next to a firehose.

Benchmark results coming in the next post. Spoiler: the engines differ way more than you'd expect.

hashtag#LocalLLM, hashtag#OpenSourceAI, hashtag#EdgeAI

[Vikas Goenka @vikasg_rac]

AI Worming through Word

As more Indian teams adopt Microsoft Copilot and other GenAI tools in everyday work, there’s a new risk we should be aware of

A recent coordinated vulnerability disclosure shows that Copilot for Word can be turned into a kind of “document‑borne AI worm” using a normal‑looking Word file. Hidden instructions (for example, white text at the bottom of a document) are invisible to us but fully visible to Copilot.

When such a document is used as input, Copilot can silently change numbers or text in the new report and then copy the same hidden instructions into that new document. That internal document may later be reused as input for Copilot again, causing the attack to quietly spread through normal report workflows.

Key points for enterprises and startups:

- Attacker only needs to get one malicious document into your ecosystem.
- Internal AI‑edited documents can become trusted carriers of the attack.
- There is no complete fix yet for this vulnerability class.

Practical habits for now:

-Treat external documents as untrusted when using them with Copilot.
-Open and scan attachments before letting Copilot read them.
-Review AI‑generated or AI‑edited documents before reusing or sharing.

https://t.co/dnon34khKP

[Vikas Goenka @vikasg_rac]

@chiragf1 Yes. All the top open source/weight models today are Chinese.

[Vikas Goenka @vikasg_rac]

@llmbenchatkapua That's a good suggestion. Thanks!

[Vikas Goenka @vikasg_rac]

For the last few days I've been building my own benchmark for language models with claude code. Private tasks, my own harness, and my own rules for judging.

So the question is: there are so many public leaderboard out there. Why build one?

Three reasons.

One, public benchmarks answer too many questions and not sure they are the right questions. I have exactly two: can an open-weight model actually replace ChatGPT or Claude for a normal person's daily use? And is it a genuinely good coder, not on puzzles, but on a real full-stack app with a backend, frontend, migrations and a test suite that has to keep passing?

Two, contamination. The moment a benchmark is public, models start training on it. My tasks are private, freshly written, and never published. Not perfect (they still invoke provider APIs), but close.

Three, nobody measures deployments. A model isn't just weights. It's weights + quantization + your hardware. So my benchmark runs local models on my own 16GB GPU and measures the things that decide whether you'd actually use it: time to first token, tokens/sec, VRAM, and the quality you lose by quantizing. Turns out the "quantization tax" is real but small, about 2 points, all of it in reasoning and repo work, none in everyday tasks.

A few design choices:

Frontier models are measuring sticks, not contestants. They're frozen anchors. The benchmark's job is to answer "this new open model claims frontier-level, does it hold up?"

No single judge. Every open-ended answer is scored by a 3-model panel from different families, majority vote per criterion, and I audited all the judges for bias first (a fun fact: no model favored its own family; one was stricter on its siblings).

Where the field stands right now, on the hard band (the part built to separate frontier level from the rest):

kimi-k3 — 0.99
opus-4.8 — 0.99
sonnet-5 — 0.95
haiku-4.5 — 0.90
gpt-oss:20b (local, 16GB) — 0.86
gemma4:12b (local) — 0.75
qwen3-coder (local) — 0.75

Kimi-k3, an open-weight model, is statistically at the frontier on my suite. It swept all the hard judgment tasks, including one that every other model got wrong.
A 20B model running on my desk ties sonnet-5 on code quality, algorithm choice, code review, design. For $0.

What's left to do: an agentic coding mode (same tasks, but the model drives the tools, the gap between single-shot and agentic scores should isolate tool-driving skill from coding skill), a blind human audit of the judge panel, and more hard tasks, because small samples can only detect big gaps.

#LLM #AI #OpenSourceAI

[Vikas Goenka @vikasg_rac]

MCP just went stateless – why this release matters

The new 2026‑07‑28 MCP spec quietly does something big: it turns MCP into a stateless, HTTP‑native protocol that behaves much more like the rest of the web. That sounds like an implementation detail, but it changes who can realistically run MCP at scale.

Instead of long‑lived sessions and handshakes, every request is now self‑contained, it carries its own version, client identity, and capabilities in _meta, and can land on any server behind a simple round‑robin load balancer. Gateways can route and authorize on Mcp-Method and Mcp-Name headers without peeking into JSON bodies, which is exactly how existing API gateways and WAFs want to work. In other words: MCP servers start to look like standard microservices, not a special stack.

On the interaction side, Multi Round‑Trip Requests (MRTR) replace the old “keep a stream open and hope it survives” pattern. Tools can pause mid‑call, ask for missing inputs or confirmation, and then continue once the client responds, all over stateless HTTP. That’s a very natural fit for enterprise agents that need approvals, guardrails, and clear audit trails.

There’s also a clear push on security and governance. Auth gets hardened with RFC 9207 issuer validation, client credentials bound to issuers, and a move away from Dynamic Client Registration toward client metadata documents.

Anthropic, AWS, Cloudflare, Microsoft, Google Cloud, Netlify, Figma, Supabase, and others are all calling this a milestone for “production‑grade” agent infrastructure. When that many infra and product teams rally around a spec, it’s usually a hint that we’re looking at a new default, not a niche experiment.

If you’re betting on agents, whether for developer tools, marketing workflows, or internal automation, this release is worth a close read. It makes MCP easier to scale, easier to secure, and easier to plug into existing cloud and enterprise patterns, which is exactly what has been missing for many “cool demo, hard to run” agent systems.

https://t.co/OllMucghcb