Kris D.

Research Fellow @ @UAlbertaCS alum interested in AI, robotics, and penguins—dabbles w/ game dev, pixel art, speedrunning, and speedcubing.

[Kris D. @M33pinator]

@HamidMaei @ninaddaithankar @AlexiGlad @ylecun @RichardSSutton I had similar thoughts where it sounded like GVFs for vision. On closer inspection, they learn to predict latent residuals Δz such that z' = z + Δz, and the claim seems to be that using temporal prediction can replace augmentations usually needed to learn "good" visual features.

[Kris D. @M33pinator]

@leggedrobotics Do you have ablations leaving each factor out to assess the contribution of each?

[ROBOTIS AMERICA @robotisamerica]

We are excited to see researchers and students continue pushing open-source robotics forward with Open Ant 🐜
The project uses XC430 and XM430 actuators to bring agile locomotion to life. Proud to support innovation in robotics education, research, and biomimetic systems worldwide.
Explore the Openmind Research Institute Github for more information - https://t.co/RbZbGJtJLD

[Richard Sutton @RichardSSutton]

The bitter lesson in 26 words:

Don’t be distracted by human knowledge, as AI has been historically.
Instead focus on methods for creating knowledge that scale with computation, like search and learning.

[Beijing Daily @DailyBeijing]

A tailor-made "kindergarten" for robots. Turing Award winner and a principal founder of reinforcement learning, Richard Sutton, has signed an agreement with a #Beijing-based company, collaborating to build a "Robot Kindergarten", planned to be located in Shougang Park in Beijing.

[sorina @robot_in_space2]

The RL benchmark "Ant" is no longer confined to simulation.

Our OpenAnt paper, to appear at @RL_Conference, is an accessible, open-source platform for RL researchers.

Here's a video of it continuously learning a back-and-forth task from experience.

Joint work with Patrick Spieler (@Stapelzeiger), Khurram Javed (@kjaved_), Kris De Asis (@M33pinator), John D. Martin (@jdmartin86), Martha Steenstrup, and Joseph Modayil (@JosephModayil)

[RL in Big Worlds @rlc_bigworlds]

RL in Big Worlds is a workshop at @RL_Conference about ideas that enable agents to achieve goals in environments vastly more complex than themselves

This requires giving agents the ability to learn continually and use approximate value functions, models and policies effectively

[sorina @robot_in_space2]

We organized an RL competition during the first Openmind Research Institute Winter School in Malaysia. The participants were able to implement SARSA and SAC in just 2 days onboard our Embodied MuJoCo Ant! 🎉

[Yu Su @ysu_nlp]

The bitter lesson of 2026 will be sim2real is hopeless to solve and the real world is the only viable learning playground.

Evolution is about overfitting to (niches of) the world. slight deviation in a simulation leads to a different universe.

[Kris D. @M33pinator]

@crypticcracking To be fair, these are only considering language models. We lost this race to SAT solvers long ago. :')

[Kris D. @M33pinator]

@kunlei15 This might be messier in that one- vs. multi-step is already common terminology in RL (e.g., n-step returns, TD(λ), etc.). What you're describing has been previously defined as the granularity of generalized policy iteration (4.6 in Sutton & Barto)

[Khurram Javed @kjaved_]

I wrote a thing. Current humanoid robotics startups are not ready for the messiness of the world.

Even if they succeed at everything they believe they need to do, it would still be insufficient for making useful robots.

More here: https://t.co/kkCIuEvm2N

[Kris D. @M33pinator]

@JoshPurtell Yeah conceptually a lot of SGD-like updates handle this by just non-stop updating, but there are practical issues with non-convex functions and their loss landscapes, where a trained network is a horrible initialization for new data—see loss of plasticity (Dohare et al., 2024) 😅

[Kris D. @M33pinator]

@JoshPurtell It’s continual* learning, and it’s learning with a non-stationary data distribution (Abel et al., 2023; Elelimy et al., 2025). It’s also not separate from RL—there’s a body of work on continual RL. Because dynamic programming is inherently non-stationary, RL is no stranger to it.

[Kris D. @M33pinator]

@marcodisarra @KhurramJaved_96 If the robot was designed to learn, then by design, it might not even have a concept of tripping and falling. Those concerns are if one tries to force a learning algorithm onto a robot designed for functional capabilities. Key is that software and hardware must co-adapt here.

[Kris D. @M33pinator]

@Tha_JPo I do think world models are important, and think they're complementary to online learning—e.g., an online-learned or continually, online-refined model. They allow directed exploration, better credit assignment, and decision-time planning—partially addressing sample efficiency. 😃

[Kris D. @M33pinator]

Reinforcement learning 🧠 on robots 🤖 can’t stay in simulation forever. My new post explores why direct, on-hardware learning matters and how we also need smarter mechanical design to enable it.

https://t.co/0GKl8naFJ8

[Kris D. @M33pinator]

@KnightNemo_ While not stated, I do think model-based RL (with decision-time planning) will be central for directed exploration, which also reduces the burden on hardware. 😃

[Kris D. @M33pinator]

@bern_jaeger @KhurramJaved_96 That's fair, and I agree that the other approaches are really useful. I'll note however, that the strong claim is rooted in the blue-sky robotics motivation rather than well-characterized, immediate practical uses. :')

[Kris D. @M33pinator]

@bern_jaeger @KhurramJaved_96 Is it really a flaw? It's true that a) it has been a criticism of direct-hardware learning, and b) that direct-hardware learning also addresses it, regardless of whether domain randomization handled it. A plus is a practitioner won't (ever) have to specify a domain distribution.

[Kris D. @M33pinator]

@adam_patni @KhurramJaved_96 Source, if curious😉 https://t.co/Tohf1eT4Hk

[Mohamed Elsayed @mhmd_elsaye]

Real-time robot learning in action @RL_Conference! The robot is built by Sorina Lupu (@robot_in_space2) & Patrick Spieler. It uses a streaming DRL algorithm (Stream AC) to learn how to walk from scratch under 10 minutes, given enough friction with the floor.