We have achieved fully non-blocking decentralized training on a recurrent model, within 0.6% of centralized quality. To our knowledge, a worldwide first.
In plain terms: training AI across distributed GPUs normally forces a choice. Either the GPUs pause and wait to sync with each other (slow, expensive) or you skip the sync and quality drops. We just showed you can have both. No blocking, no meaningful quality loss.
We chose the hardest test case on purpose. Recurrent models are sequential by nature, every step depends on the last. Transformers are far easier to parallelize. If our approach holds on the hardest case, the easier architectures should follow.
To our knowledge, no one has published decentralized non-blocking training for a recurrent architecture before. Parallax is the first. This is new ground.
Only on Chutes.
$TAO
compute is a commodity, inference is a commodity, but so is signal, information, and intelligence.
Imagine markets not measured as flops or tokens-per- second, but KL divergence, RL reward, and loss reduction.
Reliquary is live on Bittensor Subnet 81.
A market where miners are paid to find the prompts a model is about to learn from — and every rollout is cryptographically verified before it trains the next checkpoint.
@DrocksAlex2 Same, I’m building a fully encrypted AI agents platform for companies on top of Chutes. The results are mind-blowing, it can generate very well designed pdf, websites or any other agents tasks. 15 times cheaper without compromising any data. Even with qwen3.6-27b
a decentralized network of GPUs is more privacy aligned than a classic business like OpenAi.
"I don't care about my data":
...until you will be forced to care.
"Chutes is only open-source models":
...you don't need Opus 4.7 for everything.
"Chutes is too high marketcap":
Yeah... For a good reason
SN97 Arena v3 is live.
Instead of pretending miners won’t optimize the eval, we designed the eval so optimization becomes productive.
That is the Goodhart problem in reverse. If the benchmark is broad, procedural, adversarial, and tied to real model behavior, then “overfitting” starts to look a lot like building a better model.
KL still matters, but it is no longer the center of gravity.
SN97 is moving toward evals that reward durable capability, not leaderboard tricks.
We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3.
72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely.
1/n
Affine miners be aligned like:
> training is solid across multiple strats. the board is overfit-heavy with clear lower and upper bands, so we’re prepping both paths: general update-proof models for new envs, and targeted overfit on fixed-len via sft/grpo/dpo; we built an auto-eval pipeline that scores every checkpoint to catch gains and reversals. we’re also brute-solving unsolved tasks to compress ranges and lift the lower band, then feeding the best data into base and top models to win on the pareto frontier. we’re throttling deployments to avoid seeding fresh solves too early, confident this stack lets us lead soon with staged checkpoints. design-wise, incentives still favor overfit and affine sat/ded/abd under-detect it; geometric mean kicks out overspecialists but the landscape under-penalizes overfit; probably dynamically boosting weight on lowest-mean envs would raise innovation
What’s cool here is that they will use Affine environments so the decentralized, permissionless, economically incentivized RL network monetizes itself directly into another subnet on Bittensor
Come pioneer open intelligence with us and help us build our future.
-Research Engineers - ML/RL/Agents
-Protocol Engineers - Incentive design/validation mechanisms/RL/ML
-Subnet Community Moderator & Developer Relations
$10k referral reward if your intro becomes a hire.
We’re hiring from anywhere in the world offering very competitive pay.
Apply at [email protected]
Things have changed since running the first one. But some of the same principles live on, like:
- the miners are more cracked than you, accept it
- fix the incentives, don’t worry about anything else
- build trust, it’s the most valuable commodity
… and many more. Fuck i love this game
The Mission :: @const_reborn
Worlds First Permissionless, Incentivized, Open Source, Decentralized Training on Subnet 3 :: @tplr_ai
Novelty Search >> E037
Full episode: https://t.co/K2eZust5II