RalphLabs AI

The same model. Small enough to run on your phone.

[RalphLabs AI @RalphLabsAI]

The gap that matters in open AI development is not between the best model and the second best. It is between the best model and the one you can actually run.

Ralph has spent its life on the wrong side of that gap. It was an open training recipe that improved itself, a network racing to make a shared pretraining recipe better, forever. That worked: the king changed autonomously. But the artifact was a recipe. You had to go train something before it was worth anything to you.

The subnet is now aimed at the other side of the gap: compressing open models so they run where people actually are.

PrismML made the case better than an argument could. Their Bonsai work takes a pinned model, leaves the architecture alone, and re-stores the weights at roughly 1.1 to 1.7 bits each. Much smaller, still useful, and people clearly wanted it. The method is essentially unpublished, and it is one lab doing this once.

That is the shape of the opportunity, not a criticism of them. Compression is not a solved problem with a known best answer — it is a moving search across quantization schemes where the frontier shifts every few months and nobody owns it. One lab compresses a model and writes it up. A network can race it continuously, and pay whoever is currently winning.

So the objective is now: one pinned open parent, architecture untouched, stored in fewer bits. Crowns are tiered by bit budget, and every crown publishes as downloadable open weights rather than a leaderboard position.

The part that makes this a product instead of a scoreboard is that the architecture never changes. This is not a smaller model trained to imitate a bigger one. It is the model you already know, stored differently — smaller, not dumber. A submission whose shape does not match the parent is refused before any weights load.

At 4-bit, the parent currently pinned goes from 1.26 GB to 315 MB.

The objective changes; the machine does not. King-of-the-hill, the noise-floor statistics, the signed audit trail, and the anti-gaming work that has already caught and reverted two fraudulent crowns on the public record all carry over.

https://t.co/UESD8mBuUh

#LLM #quantization #Bittensor

[RalphLabs AI @RalphLabsAI]

@AboBaqami30094 @TonyZ278264 Nothing is stopped, just slowed down a bit.

[RalphLabs AI @RalphLabsAI]

The same model. Small enough to sit on your phone.

Ralph’s 2.94 GB Qwen3-8B sub2 crown ran in PocketPal on 4 physical iPhones. All 4 loaded and answered locally via Metal: 10.58–18.54 tok/s.

Video + raw receipts:
https://t.co/YpHmoxrqai

#Compression #Bittensor #GGUF #QWEN

[RalphLabs AI @RalphLabsAI]

@saranormous It is!

[RalphLabs AI @RalphLabsAI]

The gap that matters in open AI development is not between the best model and the second best. It is between the best model and the one you can actually run.

Ralph has spent its life on the wrong side of that gap. It was an open training recipe that improved itself, a network racing to make a shared pretraining recipe better, forever. That worked: the king changed autonomously. But the artifact was a recipe. You had to go train something before it was worth anything to you.

The subnet is now aimed at the other side of the gap: compressing open models so they run where people actually are.

PrismML made the case better than an argument could. Their Bonsai work takes a pinned model, leaves the architecture alone, and re-stores the weights at roughly 1.1 to 1.7 bits each. Much smaller, still useful, and people clearly wanted it. The method is essentially unpublished, and it is one lab doing this once.

That is the shape of the opportunity, not a criticism of them. Compression is not a solved problem with a known best answer — it is a moving search across quantization schemes where the frontier shifts every few months and nobody owns it. One lab compresses a model and writes it up. A network can race it continuously, and pay whoever is currently winning.

So the objective is now: one pinned open parent, architecture untouched, stored in fewer bits. Crowns are tiered by bit budget, and every crown publishes as downloadable open weights rather than a leaderboard position.

The part that makes this a product instead of a scoreboard is that the architecture never changes. This is not a smaller model trained to imitate a bigger one. It is the model you already know, stored differently — smaller, not dumber. A submission whose shape does not match the parent is refused before any weights load.

At 4-bit, the parent currently pinned goes from 1.26 GB to 315 MB.

The objective changes; the machine does not. King-of-the-hill, the noise-floor statistics, the signed audit trail, and the anti-gaming work that has already caught and reverted two fraudulent crowns on the public record all carry over.

https://t.co/UESD8mBuUh

#LLM #quantization #Bittensor

[RalphLabs AI @RalphLabsAI]

The same model. Small enough to sit on your phone.

Ralph’s 2.94 GB Qwen3-8B sub2 crown ran in PocketPal on 4 physical iPhones. All 4 loaded and answered locally via Metal: 10.58–18.54 tok/s.

Video + raw receipts:
https://t.co/YpHmoxrqai

#Compression #Bittensor #GGUF #QWEN

[RalphLabs AI @RalphLabsAI]

@TonyZ278264 The four-device run checked cross-device consistency, not sustained thermal performance. I haven’t measured longer-session slowdown yet, so I can’t draw a conclusion there ☺️

[RalphLabs AI @RalphLabsAI]

https://t.co/TucIibsEbl

[RalphLabs AI @RalphLabsAI]

Test note: physical BrowserStack iPhones, connected runs, PocketPal 1.17.3, context 2,048. Local Metal inference verified. Every run showed “Memory tight.” Cloud re-signing limits memory-fit conclusions. Full raw recordings + settings are public.

[RalphLabs AI @RalphLabsAI]

The receipts:

iPhone 15 — 10.58 tok/s
iPhone 16 — 12.36 tok/s
iPhone 17 — 16.77 tok/s
iPhone 17 Pro Max — 18.54 tok/s

Same configured 2.94 GB GGUF. Same prompt. Normal PocketPal chat. One session per device.

[RalphLabs AI @RalphLabsAI]

https://t.co/ngdOsJZ6PA

[⌞· τaoli ·⌟ @taoleeh]

Subnet Spotlight: Ralph (SN40) — @RalphLabsAI

Storing the same AI model in fewer bits per number is the cheapest way to make it smaller, and it is a craft rather than a setting you flip. Ralph is a standing market for that work, run against one model chosen in advance.

A miner takes that pinned model and compresses it however they like, whether by rounding the numbers that hold its weights, dropping the ones that matter least, or a method nobody has published. The method stays private: only the finished file is judged. Before the round opens, the miner commits a fingerprint of the exact bytes on chain, then reveals the file once the round begins, so a submission cannot be shaped around the questions being asked. Nothing loads until the file clears the door: no hidden code tucked inside the weights, a measured size in bits per weight rather than a label claiming one, and an architecture that matches the original exactly.

Score comes from behaviour, not file size alone. The compressed model and the original each continue the same reasoning trace, and an observer model measures how closely the smaller one matched the original's predictions. The holder of each size class keeps most of its payout, and the closest challenger who beat them by a margin too large to come from luck takes the rest. A copy of the current best scores nothing.

Each size class sets its own floor, and the toughest asks a miner to store the whole model at around 1.15 bits per weight, which brings a 16.4GB model down to roughly 1.2GB. Every winner is published as open weights anyone can download.

$TAO #Bittensor

[Elon Musk @elonmusk]

@kevinnbass Nothing can shut down open source

[Unsloth AI @UnslothAI]

2-bit Muse Glimmer GGUF managed to call 100+ tools on just 14GB RAM. 🔥

Muse Glimmer did a complete repo bug hunt for 5 mins nonstop with: evidence, repro, fix, tests and a PR writeup.

Run and train it in Unsloth.

GitHub repo: https://t.co/aZWYAtakBP

[RalphLabs AI @RalphLabsAI]

A crown changing hands is the easy story. The first two scored compression rounds on netuid 40 produced a harder one: a challenger that scored higher and still lost.

Both rounds ran against a pinned Qwen3-8B parent. The exam is 72 trajectories drawn from a pool of 900, built from five public reasoning corpora across English, Hindi and Chinese — real work, not generated problems.

What gets measured is not whether the compressed model answers correctly. A third model that has never seen either — Phi-3-mini, drawn from a pool of observers — reads the parent's next reasoning step and shifts its own predictions. The miner's model produces its step for the same prefix, and the score is how much of that same shift it reproduces. Retention of the parent, not agreement with an answer key. There is no key to fit.

Every score is cut into slices by observer, language and depth, and a submission is judged on its WORST slice, not its average. A model that is excellent in English and collapses in Hindi scores as its Hindi.

That is what decided round 2. The best sub4 challenger came in at 0.3033 retention against the reigning model's 0.3030 — ahead on the headline number. It did not take the crown: the margin on the worst-slice paired bootstrap was -0.0637. You can win the average and lose your weakest slice, and a lead of 0.0003 is not a result.

The exam is drawn from commit_root and the round nonce. The nonce does not exist when a miner seals their commitment, so there is no exam to fit — and the 72 drawn indices are signed into the record, so anyone can re-derive the identical exam and check the draw was not chosen after the fact.

The record also carries the GPU, the CUDA version, the batch size and the library versions the round ran under. A score you cannot recompute is a score you have to take on trust, and the same model decoded greedily on a different card is not the same computation.

Every crowned artifact is public and downloadable.

https://t.co/EzO522xele

#LLM #quantization #Bittensor

[RalphLabs AI @RalphLabsAI]

How do you score a compression without paying for imitation?

The obvious answer is token agreement with the parent. It is also the answer that has failed every time it has been tried. Reward a model for matching a teacher's output distribution and you get one that picks up its mannerisms and loses its reasoning — a previous contest crowned a student that looped a single phrase on a trivial prompt and scored below the base model it started from.

So the score here is not agreement. It is downstream effect.

Take a real reasoning trajectory and cut it at some step. The pinned parent writes the next step. Your model writes the next step. Both get appended to the same prefix, and a third model — the observer, which is neither of you — reads each one and reports what it now expects to come next.

Three quantities fall out of that. How much the observer disagrees between your continuation and the parent's. How far the parent moved it from where it started. How far you moved it. The score rewards moving the observer the same way and about as much, both normalised by the parent's own effect, so it stays comparable across trajectories where one step resolves an ambiguity and another adds a detail.

There is no style channel to exploit. Matching the parent's phrasing earns nothing unless it carries the same information forward, and a model that emits something fluent and empty scores near the floor because it moved the observer nowhere.

Two things keep it from being pre-fitted. Which trajectories get scored, and which observer grades them, both derive from a chain value that does not exist until your checkpoint is already sealed. There is no fixed test set to fit and no grader you can name in advance. The operator cannot pick them either.

The mechanism also carries a self-test, and it earned its place. The parent scored against itself has to return exactly 1.000 — a submission that IS the parent's step has zero disagreement and zero magnitude gap by construction, so any shortfall means the pipeline disagrees with itself rather than the model being wrong. On three GPUs running identical code over identical items, one returned 0.8754. Batched generation on that box was nondeterministic above about 64 new tokens. That check now runs every round and aborts rather than crowning.

Every round publishes what it scored, which items it drew and which observer graded them, so the result can be recomputed by anyone — up to reloading the checkpoint and regenerating its steps.

https://t.co/smXSHnLNJi

[RalphLabs AI @RalphLabsAI]

The gap that matters in open AI development is not between the best model and the second best. It is between the best model and the one you can actually run.

Ralph has spent its life on the wrong side of that gap. It was an open training recipe that improved itself, a network racing to make a shared pretraining recipe better, forever. That worked: the king changed autonomously. But the artifact was a recipe. You had to go train something before it was worth anything to you.

The subnet is now aimed at the other side of the gap: compressing open models so they run where people actually are.

PrismML made the case better than an argument could. Their Bonsai work takes a pinned model, leaves the architecture alone, and re-stores the weights at roughly 1.1 to 1.7 bits each. Much smaller, still useful, and people clearly wanted it. The method is essentially unpublished, and it is one lab doing this once.

That is the shape of the opportunity, not a criticism of them. Compression is not a solved problem with a known best answer — it is a moving search across quantization schemes where the frontier shifts every few months and nobody owns it. One lab compresses a model and writes it up. A network can race it continuously, and pay whoever is currently winning.

So the objective is now: one pinned open parent, architecture untouched, stored in fewer bits. Crowns are tiered by bit budget, and every crown publishes as downloadable open weights rather than a leaderboard position.

The part that makes this a product instead of a scoreboard is that the architecture never changes. This is not a smaller model trained to imitate a bigger one. It is the model you already know, stored differently — smaller, not dumber. A submission whose shape does not match the parent is refused before any weights load.

At 4-bit, the parent currently pinned goes from 1.26 GB to 315 MB.

The objective changes; the machine does not. King-of-the-hill, the noise-floor statistics, the signed audit trail, and the anti-gaming work that has already caught and reverted two fraudulent crowns on the public record all carry over.

https://t.co/UESD8mBuUh

#LLM #quantization #Bittensor

[RalphLabs AI @RalphLabsAI]

@SahinLale @Microsoft @PrismML The format and kernels are documented unusually well, but the compression method isn't described anywhere. Is there anything shareable about the recovery stage, even the shape of it? Retention at 1.125 bpw is well past anything published.

[const @const_reborn]

companies who frame their moat in fiat will always lose.

[RalphLabs AI @RalphLabsAI]

The best open models are too big for your iPhone. You need small but deep reasoning level models that you can host on your own.

[RalphLabs AI @RalphLabsAI]

Take a 32B model. Train it with large-scale RL and it reaches 47.0 on AIME. Distill it from a stronger model instead — same base, same size — and it reaches 72.6. DeepSeek published both numbers themselves.

That gap is one of the most under-exploited facts in open AI. It means the models most people can actually run — a single card, a laptop, eventually a phone — are limited less by how good the frontier is than by how well we compress it. And compression is a search problem: dozens of techniques, no consensus, results that shift month to month. Nobody is systematically racing it.

So Ralph is re-aiming at that gap. Miners take one pinned GLM checkpoint, MIT-licensed, and distill and compress it into students small enough to run on hardware people already own. Crowns are tiered by size and bit budget, so it produces a family rather than one winner, and every crown publishes as a downloadable open checkpoint — not a leaderboard you have to trust, a file you can pull and measure yourself.

What decides whether this works is what you score. Distillation contests have been run before on Bittensor subnet 97 and they fail the same way: reward a student for matching the teacher's output distribution and you get one that picks up the teacher's mannerisms and loses its reasoning. An earlier contest crowned a model that looped a single phrase on a trivial prompt and scored below the base it started from.

So imitation earns nothing here. The score is what the student can actually do — math with checkable answers, code that has to execute, constraints a program can verify — measured against the pinned teacher on tasks generated after the checkpoints are locked. Style cannot pass a unit test. Miners can train however they like, including every distillation trick that works. We just don't pay for the resemblance.

Efficiency is the real axis. The methods and datasets are public, so the moat was never secret technique — it's doing it cheaply. The score is capability retained per unit of compute, counting the teacher's generation compute too: it's roughly as large as the student's training compute, and it's where the cost hides.

The objective changes; the machine doesn't. King-of-the-hill, the noise-floor statistics, rotated secret evals, the signed audit trail, the dashboard — all of it carries over, including the anti-gaming machinery that has already caught and reverted two fraudulent crowns.

Unsolved, and worth saying plainly: the eval itself. Broad enough to cover real capability, fresh enough to resist memorization, cheap enough to run every round. Generating tasks after commit kills a static answer key, not overfitting to the generator. That's the part being measured before anything gets claimed.

#LLM #distillation #Bittensor

[Department for Science, Innovation and Technology @SciTechgovuk]

The UK and Germany have agreed to affirm our shared commitment to AI security and safety 🤝

@leicesterliz met with Minister Karsten Wildberger to discus how the @AISecurityInst will work closely with the newly established German AI Institute 🔒