jpmonty

I help machines understand the world.
Views are my own.

[jpmonty @jpmontoya271]

@mauro_ai MuJoCo + blender

[jpmonty @jpmontoya271]

I made my humanoid walk in my virtual kitchen using a pretrained walking policy.

Getting to my stove should not be this hard.

My kitchen, recreated in MuJoCo. Walking policy from Lucky Robots’ G1 Manipulation Challenge⁠.

[jpmonty @jpmontoya271]

@tv0ff I haven’t tested sim2real transfer yet.
Need a bot.

[jpmonty @jpmontoya271]

I put TypeSafe’s JEV in control of G1 humanoid in my virtual kitchen.

The mission: reach the stove with obstacles on the way.

A VLM and depth information helped map the kitchen.

A path planner produced possible ways around the obstacles, and JEV chose which move to make next.

A pretrained walking policy handled locomotion, with calibrated controls for turns, forward movement, and curves.

The robot reached the stove in about 64 seconds of simulation time. This video shows the highlights.

Still a prototype, but pretty cool watching perception, decisions, and movement come together.

[jpmonty @jpmontoya271]

@robixone I will start testing with non static obstacles, will keep you posted.

[jpmonty @jpmontoya271]

@0fir0z Qwen3.8 generates a scene description.

Then scene description plus potential trajectories is passed to JEV.

Finally JEV makes trajectory decision.

Trajectory runs a pretrained policy for locomotion.

[jpmonty @jpmontoya271]

@lmdrezende Qwen3.8

[jpmonty @jpmontoya271]

Humanoid learned to open the fridge.

[jpmonty @jpmontoya271]

@msn_muhn Mujoco -> Blender

[jpmonty @jpmontoya271]

My roommate told me to wash the dishes.

So I asked a coding agent to build a MuJoCo simulation of my kitchen to train a humanoid to do it.

[jpmonty @jpmontoya271]

@Robotdarkg1d Coding agent -> Python -> MuJoCo -> Blender

[jpmonty @jpmontoya271]

The humanoid can now load a plate into the dishwasher in a physics simulation of my own kitchen.

Programmed arm motion, contact-based grasp, and an invisible support keeping the robot upright.

Balance is next.

[jpmonty @jpmontoya271]

@davidfeldt Did you buy one?

[jpmonty @jpmontoya271]

@ariwasch ELLY

[jpmonty @jpmontoya271]

Data is the real bottleneck.

Foundation models are improving at general tasks because those domains have abundant training data.

Using them for domain-specific workflows is inefficient by design.

They’re optimized for user satisfaction, not accuracy.

The next leap won’t come from larger models, but from allowing domain experts to curate better datasets, and contribute to foundation model training.

[jpmonty @jpmontoya271]

@Sajeel_Purewal This is really impressive!

[jpmonty @jpmontoya271]

@hthieblot Only 38 tho 😂

[jpmonty @jpmontoya271]

@MrAMNC Fortune cookies don’t lie

[jpmonty @jpmontoya271]

@sama 🤔

[jpmonty @jpmontoya271]

@sincethestudy These robots low key have better quality and higher budget that some real products out there 😂