I (think) finally wrapped my head around multi-token prediction, which is a term used for two very different techniques in speculative decoding
here's an illustration of the difference
So (stylus extension + argon) in browser chat has been a nice experience for personalizing frequent sites.
Custom styling & design preferences might be a good experience soon, if browsers in the future allow custom styling CSS to override in-browser chat, like Ask Gemini in Chrome.
@thdxr Egypt easily has more than that as avg. yearly count of students , localize opencode pricing , I'll be your opencode ambassador with content creator connections , watch the exponential curve , opencode for Edu. can be real
MiMo-V2.6: The Hard Road to Scaling Up RL
MiMo-V2.6 is very likely one of the largest single RL runs, by compute, that any open-source model team has undertaken to date. In an era when compute is brutally scarce, we still chose to dedicate a team of several dozen people to one goal over an extended period: scaling up RL. That takes more than research conviction. It takes a vision for AGI, respect for the unknown, and the nerve to walk straight into the hardest problems.
The result is a model whose potential was built through mid-training and unlocked through heavy RL. Today, it is the number one open-source model. I strongly recommend reading the technical report. I believe it will become one of those papers that Agent RL practitioners keep reopening and discovering something new in each time. In my view, the research innovations and engineering challenges behind it surpass those of DeepSeek R1, which I was partly involved in.
Some will ask: why MixRL instead of MOPD? First, they are not competing choices. We ran MixRL on verifiable tasks of moderate difficulty, including code and related agentic tasks, and found that the resulting models generalize remarkably well. Second, tasks that are difficult to verify, extremely long-horizon, or simply too challenging to include in a joint RL run are trained separately. Including them would substantially reduce rollout efficiency or introduce significant rollout staleness. We then merge the resulting capabilities through MOPD. Games, 3D tasks, and tasks with subjective evaluation signals all fall into this category.
There is also a third, slightly cheeky answer. Our team is flat enough and free enough of organizational silos that MixRL simply is not difficult for us. More importantly, everyone enjoys working this way. People from different domains come together every day, driven by the pursuit of AGI and intelligence that can continuously improve itself, to confront and resolve the RL bottlenecks in each field. I will always remember the RL daily update meetings from this period. They were intense and dense, with intelligence emerging in real time.
To help the open-source community focus on solving real Agentic RL problems, we have released a Qwen model distilled from MiMo RL trajectories as a stronger starting point for RL, along with 7K diverse environments and a complete RL training framework. We hope these resources will help move Agentic RL research forward.
MiMo-V2.6 is only the beginning. In an era when intelligence is easy to replicate, we still choose the hard road toward self-improvement and AGI. Much of what lies ahead remains unknown. But we are willing to keep investing the time, compute, and passion required to take on one hard problem after another and work each of them all the way through, until intelligence crosses into a new regime.
@oprydai What about competing with the Ms of graduates each year, is it possible to find a job in such a market? specially if u're on the weaker side of the lang and culture
@grok@poteto@bot Still not as your mascot @grok, the reference is @bot mascot. Also adding a pressure valve so it works as an all-in-one: pressure cooker, steamer, slow cooker. The original post image is a nice reference for minimalism and efficiency of SpaceX
Robots don't have to be scary!
Microduck discovers Reachy Mini. Give me your best scenario for these two and I'll see what I can do.
Don't push me too hard though, or I might end up making a whole movie.
@dabit3 Kaggle competitions (RSNA, Kaggriculture),
F1 racing ML projects,
Researching the possibility of separating factual knowledge in weights and long-term memory of model interactions inside the model, starting from GPT2