Jeanius

strategic partnerships & ops | building the hub for all things reinforcement learning

[Jeanius @__jeanius]

@grok

[SiHing Guppy @sihing_guppy]

Lighting changes constantly: time of day, weather, different rooms, sensor drift. If a model only works under the lighting conditions it saw in training, it has not really learned the task. It has learned one appearance regime.

We put this to the test. Two models that take language instructions and turn them into robotic actions, Pi 0.5 and SmolVLA, ran the same manipulation tasks on a standard benchmark (LIBERO-Spatial) while we shifted brightness, exposure, gamma, contrast, saturation, white balance, and color temperature.

Same geometry, same objects, same tasks. Only appearance changed.

Pi 0.5 barely moved. Across nearly every perturbation, even at the highest severity, it stayed within a few percentage points of baseline. The only measurable dip was contrast, to around 94% of baseline. Not a collapse. A graceful decline.

SmolVLA degraded under nearly every one. Saturation cut performance roughly in half. Brightness produced steady losses. Even gamma, white balance, and color temperature caused visible degradation. And then there was low contrast. SmolVLA went from baseline to near-zero. Not a degradation curve. A complete collapse.

If both models had broken, you could argue photometric robustness is just hard, something inherent to vision encoders. Pi 0.5’s near-total immunity rules that out. Photometric robustness is achievable. SmolVLA’s failure is diagnostic.

The pattern suggests SmolVLA is much more dependent on the appearance statistics of its training data. Many models silently use color as a shortcut for object identity, affordance, or state. When color shifts, those shortcuts break.

By contrast, Pi 0.5 appears to have learned much stronger invariance to lighting and color shifts. Training augmentation is likely part of that story.

The two models do share one vulnerability: low contrast.

Pi 0.5 dips gently. SmolVLA collapses.

That likely reflects something deeper about how vision encoders extract features. When edge contrast drops too far, the gradients driving feature extraction weaken, and downstream representations lose the structure needed for precise action prediction.

Standard augmentation pipelines also rarely suppress contrast as aggressively as real-world conditions can.

If a model fails when the lighting changes, it has learned the lighting conditions of the demo, not the task itself.

Full analysis with interactive visualizations:
https://t.co/NJvEl0acBg

[SiHing Guppy @sihing_guppy]

https://t.co/Lw3jTrEtgN

[SiHing Guppy @sihing_guppy]

What happens when you remove a robot's ability to read its instructions? Almost nothing.

> Full model → 95% success
> Remove language → 94% (▼1%)
> Remove vision → 13% (▼82%)

Near-blind without vision. Near-indifferent to language.
If your evaluation only tests correct instructions, you're not measuring language. You're measuring vision.

[SAI @CompeteSai]

Knowing where your system will break is just as valuable as knowing where it works.

[SAI @CompeteSai]

It wouldn’t be robot soccer without the replays.

Here are our favourite moments from the Booster Soccer Showdown's top contenders.

Grateful to everyone who built and competed with us this season.

[Jeanius @__jeanius]

@CompeteSai @boosterobotics Vienna, the SAI community is coming for you!!!

[SAI @CompeteSai]

SAI is hosting an official ICRA 2026 competition: the Sim2Real Soccer Challenge, in partnership with the University of Freiburg and @boosterobotics.

Teams will train policies in the SAI Soccer Simulation Suite and deploy those same policies on Booster K1 humanoids during the finals in Vienna.

A unified benchmark for sim-to-real humanoid control.

[SAI @CompeteSai]

At @UofT last week, we focused on one thing: giving undergraduate students a real, working understanding of reinforcement learning.

From mapping out the RL loop to reward engineering to training and submitting a model on SAI, the workshop gave students the kind of applied entry point we wish we had when first starting out.

SAI was built to showcase RL talent, but it also exists to build the next generation, and access like this is where it starts.

[SAI @CompeteSai]

A stutter-step in sim.

The first working model just landed in the Booster Soccer Showdown, built by Maub, now sitting at #1 on the leaderboard.

There's still 52 days to go in the challenge. Who’s adding the first body feint next?

[SAI @CompeteSai]

SAI is growing with the open-source community.

You’ll soon be able to submit models directly from your @huggingface account.

Connect, tag, and benchmark your work alongside the global RL community.

One more step toward an open, shared ecosystem.

[Jeanius @__jeanius]

@NRNAgents @yellowpantherx every yp needs a robot to match its energy

[SAI @CompeteSai]

Three tasks. One policy.

Registration for the Booster Soccer Showdown is live.

Benchmark your agent across multiple tasks.

Presented by @CompeteSai · Sponsored by @BoosterRobotics

Register today -> https://t.co/9AzylZBAlM

[SAI @CompeteSai]

Our Feature Focus series continues with action functions.

Have more control over how your model acts.

Sample from probabilities. Scale down joystick movement. Or even apply Monte Carlo Tree Search.

Your Model. Your Rules. Try it now on SAI.

[Jeanius @__jeanius]

@Eeazy_Weezy @NRNAgents we indeed were too scared to go up against you

[Jeanius @__jeanius]

@aiarena_ @gam3sgg @HotLikeLava_ bullish about @HotLikeLava_

@gam3sgg can i nominate this nomination for the most well deserved nomination? (say this 5x then respond)

[Jeanius @__jeanius]

@aiarena_ how i’ll be attending team meetings for the next month

[Sam Steffanina @SamSteffanina]

WE JUST HIT $15,000 RAISED!!!

A big $2,000 donation from @aiarena_ helped us hit our biggest milestone yet

I love everyone who has donated thus far, big and small <3

[Sam Steffanina @SamSteffanina]

24h STREAM! RAISING $50,000 FOR CHARITY! https://t.co/3iSAgCK1DI

[Jeanius @__jeanius]

@aiarena_ to think of all that AI Arena has accomplished in just another year is mind boggling.

can’t wait to see where the next 365 take us. 🚀