Researchers made KMeans 200x faster.
And the new technique also beats approaches like cuML and FAISS.
Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks.
By attacking the memory bottlenecks directly, Flash-KMeans achieves:
- 33x speedup over cuML
- 200x speedup over FAISS
This speedup comes from how it moves through GPU memory.
Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory:
1) The first step matches every point to its nearest centroid.
Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck.
Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out.
2) The second step recomputes each centroid by averaging the points assigned to it.
Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn.
Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass.
Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds.
The video below depicts this in action.
Several reasons why this is important:
KMeans has always been an offline primitive. Something you run once to preprocess data and move on.
These speedups make the approach viable in several runtime-critical systems.
↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes.
↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes.
↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing.
I have shared the paper in the replies.
That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks.
I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds.
Read it below.
@gsusMad@pacotraver Gracias por compartir. Muy interesante. Me interesaría saber si probaste otros modelos además de potion-multilingual-128M y si este carga por default cuando se abre el sitio o se usa on demand.
AI will not make engineers more productive.
It will make bad engineering faster.
Here's the pattern I see:
A junior engineer asks an AI to write the simulation code.
The AI writes something plausible.
The engineer doesn't fully understand it.
The simulation runs.
The results look reasonable.
The engineer ships it.
Nobody caught the edge case in the boundary conditions.
Nobody questioned whether the model assumptions were valid.
Nobody held the equations long enough to develop intuition.
Speed without understanding is not productivity.
It's technical debt with better syntax.
The engineers who use AI well are the ones who already know enough to catch its mistakes.
The ones who don't know enough use it as a crutch and call it skill.
AI raises the floor for bad engineers.
It raises the ceiling for great ones.
The gap between them is getting wider, not smaller.
Where are you on that spectrum? Be honest.
🚨 A website can figure out what you're doing on your computer.
No download. No permission. No popup.
> It's called FROST.
> Up to 95% accurate.
> And there's no fix yet.
You just leave the tab open, and JavaScript times your SSD to tell which sites you visit and which apps you open.
🔗 Learn how this works: https://t.co/kyvObeVoR4
Which one is better?
Opus 4.6, Sonnet 4.6, or GPT-5.2-Codex?
The good news: this might not matter soon!
Because the models are commoditizing, and the real differentiator is moving elsewhere.
On general benchmarks like MMLU, frontier models have saturated to the point where there's barely any room to differentiate.
And on the agentic benchmarks that actually reflect production work, like SWE-bench and TerminalBench, what's being measured isn't the model alone.
It's model plus the infrastructure around it. On SWE-bench Pro, scaffold changes produce 22x larger swings than model swaps do at the frontier.
Meta and Harvard even ran Claude Sonnet (not Opus) with a custom scaffold and it slightly outperformed Opus on Anthropic's own scaffold. A cheaper model beat the flagship because the surrounding infra was better.
That infrastructure is what the industry now calls the agent harness.
It covers the orchestration loop, tools, memory, context management, error handling, and verification layers around the LLM.
When agents fail at multi-step tasks in most production scenarios, the failure almost always traces back to a context problem, not a capability problem.
The model didn't have the right info at the right time. And the harness is the thing that controls what the model sees, when it sees it, and what happens when something goes wrong.
If you are building an agent harness yourself, it comes down to 7 key design decisions (see the visual above).
1) Agent count is about single vs. multi-agent. Both Anthropic and OpenAI recommend single-agent first because multi-agent adds overhead from routing and loses context during handoffs.
2) Reasoning strategy is about whether the agent thinks and acts in lockstep (ReAct, flexible but costly) or plans first and executes separately (significantly faster but less adaptive mid-task).
3) Context strategy matters because model performance degrades when key content lands in mid-window positions. Claude Code handles this by preserving architectural decisions during compaction and discarding redundant tool outputs.
4) Verification is probably the highest-leverage harness investment. Computational checks like tests and linters give you deterministic ground truth, while inferential checks like LLM-as-judge catch semantic issues.
5) Permissions range from permissive (auto-approve, fast, risky) to restrictive (pause for confirmation, slower, safer). Claude Code gates around 40 tool capabilities independently.
6) Tool scoping controls how many tools are visible at any given step. More tools in context typically means worse performance. Claude Code reduces this through lazy loading, only surfacing tools when the agent determines they're relevant.
7) Harness thickness is about how much logic lives in the harness vs. the model. Anthropic bets thin, deleting planning steps as newer models internalize them. LangGraph goes thick with explicit graph control.
A general pattern across the industry is that harnesses are getting thinner as models improve, but the harness itself isn't going anywhere.
Even the most capable model needs something to manage its context, execute its tools, and verify its work.
If you want to learn more, my co-founder, Akshay, wrote a deep dive on this covering all 11 harness components and how Anthropic, OpenAI, LangChain, and CrewAI each implement the pattern.
Read it below.
____
Find me → @_avichawla
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.
@Reforma@MiguelMezaC Por favor compartan. Este video está traducido al inglés. Se necesita difusión a nivel internacional! México con este gobierno incapaz no está capacitado para celebrar ni un mundial ni nada por el estilo.
Hoy por ella… mañana por nosotros. 🙏🏻
Announcing ARC-AGI-3
The only unsaturated agentic intelligence benchmark in the world
Humans score 100%, AI <1%
This human-AI gap demonstrates we do not yet have AGI
Most benchmarks test what models already know, ARC-AGI-3 tests how they learn
I made a fast, tiny tool for quickly sharing small datasets without uploading any data to a server.
It compresses the data into the link itself, so there’s no account, hosting, or storage layer.
Here’s Florence Nightingale’s famous 📊 data as a live example (video below):