stephen

idle vapourings of a mind diseased | deploying rl @ apptronik

[Kevin Cook @TevinTook]

Excited to share more of Robot Park with the world, we've been having fun building and scaling it!

[Kyle🤖🚀🦭 @KyleMorgenstein]

Thrilled to finally share Apollo2 and Robot Park with the world! I’ve spent the last year working with Apptronik’s dynamite controls team as the RL technical lead. We’re just getting started- more to come soon! 😃🤖

[Apptronik @Apptronik]

Welcome to Robot Park, where we’re building the future with Apollo 2. Robot Park is where Apollo learns today, getting the experience needed to make a difference tomorrow.

#Apptronik #Robotics #AI #Humanoids

[ирис @saltwatermp3]

If we are ever playing chess together and you take my horse you are dead to me. I like him

[stephen @_ildsjel]

@KyleMorgenstein @carlosdponx @void_bio @yacineMTB All that stands between me and sim2real is a soldering iron

[stephen @_ildsjel]

@carlosdponx @void_bio @yacineMTB I'm using MJWarp lol bet

[stephen @_ildsjel]

@carlosdponx @void_bio @yacineMTB Fwiw I'm hitting ~2M sps for a 21 dof humanoid in Python with default hparams/simple task

[stephen @_ildsjel]

@CryptoCyberia It's just a policy switch

[BusEnjoyer01 @busenjoyer01]

Fun fact:

At 470 feet tall, the Capital One Tower in Tysons, Virginia is the tallest building in the United States.

[stephen @_ildsjel]

@JayHe748646 @AlecSaysThings Happy to chat! DM me and we can schedule something

[Chester @chesterzelaya]

the male equivalent to flowers is probably an RTX6000 Pro Blackwell Workstation

[stephen @_ildsjel]

YYYYYYIPEEEEEEE

[Yuval Tassa @yuvaltassa]

Headline new features from MuJoCo 3.7:

DC motor actuator, midpoint integrator for super stable free bodies, and improved equality constraints.

https://t.co/3fR7mK2gVy

[stephen @_ildsjel]

@trev3d Do you like robots

[stephen @_ildsjel]

My dreams come true

[Kevin Zakka @kevin_zakka]

Really excited to release mjviser, a web-based MuJoCo viewer, powered by Viser. It has almost all the features of the native MuJoCo viewer, but runs in your browser. Load and simulate any MuJoCo model with a single uv command 👇

uvx mjviser <model.xml>

[Kevin Zakka @kevin_zakka]

@KyleMorgenstein @ChongZitaZhang https://t.co/ZJgbwVwuPC

[stephen @_ildsjel]

Similar to embed2control!

[Zhuokai Zhao @zhuokaiz]

Latent world models learn differentiable dynamics in a learned representation space, which should make planning as simple as gradient descent.

But it almost never works.

What I mean is, at test time, you can treat the action sequence as learnable parameters, roll out the frozen world model, measure how far the predicted final state is from the goal, and backprop through the entire unrolled chain to optimize actions directly. Yet many of the systems that work (Dreamer, TD-MPC2, DINO-WM) abandon this and fall back to sampling-based search instead.

That's why I really like this new paper by @yingwww_, @ylecun, and @mengyer, which gives a clean diagnosis of why, and a principled fix.

The reason everyone abandons gradient descent on actions is that the planning objective is highly non-convex in the learned latent space. So instead most systems use CEM (cross-entropy method) or MPPI (model predictive path integral), both derivative-free.

CEM samples batches of action sequences, evaluates them by rolling out the world model, keeps the top-k, and refits the sampling distribution.

MPPI does something similar but weights trajectories by exponentiated negative cost instead of hard elite selection.

These work when gradients are unreliable but the compute cost is substantial — hundreds of candidate rollouts per planning step vs a single forward-backward pass.

This paper asks what exactly makes the latent planning landscape so hostile to gradients and what you can do about it.

The diagnosis. Their baseline is DINO-WM, a JEPA-style world model with a ViT predictor planning in frozen DINOv2 feature space, minimizing terminal MSE between predicted and goal embeddings. The problem is that DINOv2 latent trajectories are highly curved (when you use MSE as the planning cost you're implicitly assuming euclidean distance approximates geodesic distance along feasible transitions).

For curved trajectories this breaks badly, gradient-based planners get trapped and straight-line distances in embedding space misrepresent actual reachability.

The fix draws from the perceptual straightening hypothesis in neuroscience — the idea that biological visual systems transform complex video into internally straighter representations. So they add a curvature regularizer during world model training.

Given consecutive encoded states

z_t, z_{t+1}, z_{t+2},

define velocity vectors as

v_t = z_{t+1} - z_t

measure curvature as the cosine similarity between consecutive velocities, and minimize

L_curv = 1 - cos(v_t, v_{t+1}).

Total loss is then

L_pred + λ * L_curv

with stop-gradient on the target branch to prevent collapse.

The theory backs this up cleanly — they prove that reducing curvature directly bounds how well-conditioned the planning optimization is — straighter latent trajectories guarantee faster convergence of gradient descent over longer horizons.

Worth noting that even without the curvature loss, training the encoder with a prediction objective alone produces some "implicit straightening" — the JEPA loss naturally favors representations whose temporal evolution is predictable. Explicit regularization simply pushes this much further.

Empirical results across four 2D goal-reaching environments are consistently strong. Open-loop success improves by 20-50%, and the GD with straightening matches or beats CEM at a fraction of the compute.

The most convincing evidence is the distance heatmaps: after straightening, latent Euclidean distance closely matches the shortest distance between states, even though the model was trained only on suboptimal random trajectories.

What I find interesting beyond the specific method is that the planning algorithm didn't change. The dynamics model didn't change. A single regularization term on the embedding geometry turned gradient descent from unreliable to competitive with sampling methods.

The field has largely treated representation learning and planning as separate concerns — learn good features, then figure out how to plan in them.

This paper makes a concrete case that the representation geometry is itself the bottleneck.

This connects to a broader pattern in ML. When optimization fails, the instinct is to fix the optimizer (better search, more samples, adaptive schedules). But often the real lever is the shape of the space you're optimizing in.

Same principle shows up in RL post-training where reward landscape shaping matters as much as the algorithm itself.

Shape the space so simple optimization works, rather than building complex optimization to handle a bad space.

Their paper:
https://t.co/NLPGxqbP2x

[Kevin Black @kvablack]

oh you're using VLAs? everyone's using GRPs now. just kidding we're all on LBMs. world models are the future so we developed our own WAM. we're using DVAs. we were using UWMs but our robot caught on fire so we switched to DreamUMVLAPs. we're shipping a robot that passes butter.

[stephen @_ildsjel]

@_Stocko_ I've had trouble with contact stability when modeling rolling surfaces like this before, I had to switch to ellipsoids instead

[stephen @_ildsjel]

@ChongZitaZhang Soon: