Google Deepmind presents Mixtures of Experts Unlock Parameter Scaling for Deep RL
paper page: https://t.co/IjxzP9rrV6
The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model's performance scales proportionally to its size. Analogous scaling laws remain elusive for reinforcement learning domains, however, where increasing the parameter count of a model often hurts its final performance. In this paper, we demonstrate that incorporating Mixture-of-Expert (MoE) modules, and in particular Soft MoEs (Puigcerver et al., 2023), into value-based networks results in more parameter-scalable models, evidenced by substantial performance increases across a variety of training regimes and model sizes. This work thus provides strong empirical evidence towards developing scaling laws for reinforcement learning.
I got some beautiful fella stuff to go to work in and would like to officially join #NAFO! Have a #FellaRequest - I love pusheen, javelins and vatnik tears!! And I have pink hair. @Kama_Kamilia#NAFO expansion non negotiable!! Слава Україні!
I'm a big fan of the pomodoro technique for productivity, where I do my best impression of a tomato and sit in the dark for 48 hours before I am devoured by beings beyond my comprehension
Independence Day is the day of freedom and life for our nation, for our people. Without independence, there would be none — we know it too well from our history. Today, the will of the Ukrainian people to fight for our independence inspires the entire world. Ukraine will win 🇺🇦✌️
In many coffee houses in Kyiv:
Pink post-its are beverages and desserts that visitors have paid for as a treat for military or territorial defense members, who can come and claim any of them for free.
It’s people’s way to thank their defenders.
Battle of Kyiv Thread 1/
The Battle of Kyiv was decisive battle in Ukraine War that Russia launced invading on Feb 24, 2022. Russian assault on Kyiv was catastrophic military failure, overwhelming victory for Ukraine
Battle ended 31 March with total retreat
This is summary