林 幸一 / Koichi HAYASHI

株式会社フィジカルAI総合研究所 取締役 名大博士在籍中 製造業にフィジカルAIを浸透させるべく奮闘しています
採用中です！！！！！！！DMまで。

[LeRobot @LeRobotHF]

100,000 hours of human hands doing real work. LeRobot format, fully annotated and open.

EgoSuite-Open100K from @LightwheelAI X @huggingface: 15,000+ tasks across 15,000+ real scenes, hand + body pose and wrist cams.

Recorded across 7 environment categories and 128 scene types, from kitchens and bedrooms to warehouse floors and assembly lines.

It's open for academic research and commercial training, and streams straight from the Hub. Always great to see a dataset at this scale released openly on the Hub.

More info: https://t.co/1XNb4sWLL5

[NVIDIA Robotics @NVIDIARobotics]

Newton 1.5 is now available, making robot training at scale faster and more reliable.

This release delivers:
🔁 More parallel simulation, lower memory, selective resets
🤝 Consistent contact physics
🤖 Experimental batched GPU control
📦 Cleaner USD/MJCF imports

Get started: https://t.co/55xNscD5ub

[林 幸一 / Koichi HAYASHI @KoichiWOW]

これは結構魅力的

[Léo @LeoKharon]

NEW RESEARCH: a small transformer can actually outperform large VLAs for robot control!

@ylecun and @LerrelPinto (and others) have been at it again.

Called Patch Policy, it is a robot-manipulation method, result of a collaboration between NYU's Courant Institute, Meta-FAIR, and AMI Labs.

DEFINITIONS:
- Patch features: a vision transformer cuts the image into a grid of small squares (patches) and produces one feature vector per square; "dense" means the policy sees the full grid, all 256 vectors, with their spatial layout intact.
- CLS token (pooling): the usual alternative -> collapse those 256 vectors into a single summary vector for the whole image.
- Frozen encoder: the vision model's weights are not updated during training.
- Self-supervised: trained on raw unlabeled images by solving puzzles about the images themselves, no captions or human labels.
- Policy head: the small trainable network bolted on top that turns frozen features into robot actions.
- Block-causal attention mask: an attention pattern where patches attend freely to each other within a frame but only causally across time (full spatial context now, no future frames).
- VLA (vision-language-action model): a model that takes camera images plus a language instruction and outputs robot motor commands directly, no hand-written controller in between.
- VQ-BeT: a policy head that turns action prediction into next-token classification -> continuous actions get compressed into a discrete vocabulary, and the model picks from it like a language model picks words.
- Diffusion Policy: a policy head that generates actions the way image models generate pictures -> start from noise and iteratively denoise into a short trajectory of motor commands.

In practice, it controls a robot by feeding a small transformer the full grid of dense patch features from a frozen, internet-pretrained vision encoder (DINOv2, WebSSL), instead of the usual single compressed image token or a fine-tuned billion-parameter vision-language-action model.

The one architectural trick making it possible is a block-causal attention mask: patches attend freely to each other within a single frame but only causally across time.
The visual backbone is never fine-tuned: only a ~29.5M-parameter policy head (VQ-BeT or Diffusion Policy) is trained.
Tested in sim (Push-T, LIBERO, BlockPush, Cube) and on a real 7-DoF Franka (cable insertion at 2 mm tolerance, pen collection, tool hanging).

This paper shows that a frozen self-supervised vision encoder + a tiny head beats a fine-tuned 7B VLA.
This is LeCun's anti-scaling thesis in practice!
Patch Policy outperforms fine-tuned OpenVLA-OFT by 18% on average using ~0.7% of OpenVLA's parameters.
LeCun has spent years arguing against LLM-scale-maximalism and for self-supervised vision (DINO/JEPA), and this lands squarely against the OpenVLA / π0 / Generalist "scale a big multimodal model" camp.

This is also a good demonstration that the bottleneck is not the policy or the model size, but the visual representation.
This means the whole field is probably optimizing the wrong variable (also interesting to correlate this with Stanford's FACT project; @carlotapares et al).

Backbone quality (WebSSL/DINOv2 beat DINOv3, V-JEPA 2, SigLIP 2) predicted performance more reliably than the policy head (VQ-BeT vs Diffusion) or parameter count.
The useful visual prior already lives in a frozen internet-pretrained encoder, and the win is refusing to throw its spatial detail away. It inverts the "we need robot data to adapt the vision model" premise underneath the VLA stacks.

Standard policies pool to a single CLS token, while Patch Policy keeps all 256 patches.
Going from 256 to 4 patches drops Push-T from 0.69 to 0.51.
Cable insertion at 2 mm tolerance hits 70% vs OpenVLA-OFT's 30%.
This is the same shape of argument as the Any-ttach project (i.e. precision manipulation fails because the dominant paradigm throws away information it needs; @ChengXianyi's lab) except here the discarded information is spatial resolution rather than force.

Also worth mentioning: 11 ms vs 62 ms inference, 6.5 vs 16 GPU-hours to train, 29.5M params.
~5.6× faster inference on a frozen DINOv2 front-end, ~2.5× cheaper training, all on a single L40S.
An 11 ms policy is genuinely controllable at real-time rates on modest on-robot hardware.

It beats the big VLA on the number of demonstrations per task!
Real-robot tasks used 50 to 101 episodes; sim used 206 to 1,000 trajectories.
The recurring finding is that with the right representation, tens of demos suffice.

The encoder being frozen is actually essential.
Fine-tuning it would be worse, because it forgets the internet-scale prior.
Every experiment freezes the visual backbone; only the policy head trains.
That's simultaneously why it's cheap and why it works!

[兎耳山ルカ 公認会計士 @TomiyamaLuca]

日本企業、AIに古典的職制ヒエラルキーを持ち込みがちなのだけど
大人しくanthropicやopenaiが実践するアプローチを試した方が良い

How we built our multi-agent research system https://t.co/6c5mjOQAis

A Practical Guide to Building Agents https://t.co/sKmYs1aZbG

[Kento Kawaharazuka / 河原塚 健人 @KKawaharazuka]

🎉VLAを高速化するNeural Introspection Gatingが #IROS2026 に採択されました！

VLA自身のconfidenceを用いてKV cacheを適応的にreuse/recomputeすることで, 単純なcache reuseによる精度低下を抑えつつ, 大幅な計算量削減を実現します！

https://t.co/Fnm1PdcqnM

[Dyna Robotics @DynaRobotics]

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws:

• world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours,
• this human data scaling law implied a scaling law on never seen robot data,
• both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge

🧵

[ITmedia AI＋ @itm_aiplus]

AIエージェントの「Skills」などが標準規格化　CodexやVS Codeなど対応、Claudeは未対応
https://t.co/CVhyWwM8GY

[Movez @0xMovez]

OpenAI engineer:

"At OpenAI, 85% of our engineers are running hundreds of agents with Codex. The way you do it is graph engineering."

In 30-minutes lecture, he explains how a single engineer can now do the work of an entire team, how far this has already gone inside OpenAI, and where it goes next.

This is something you can't skip if you don't want to be left behind.

Watch it, then read the full guide on graph engineering below.

[Kento Kawaharazuka / 河原塚 健人 @KKawaharazuka]

🎉大規模な一人称視点の人間動画から、ロボット操作に必要なaffordanceを学習するVLAffが #IROS2026 に採択されました！

「どこに触れる・どう掴む・どう動かす」を統一的に学習し、人間動画だけからゼロショットでロボット操作を実現します！

https://t.co/Wqb0CkrPl9

[Léo @LeoKharon]

NEW ROBOT BENCHMARK: If your robot can do Origami, it can do anything!

Called The Robotic Origami Challenge, it is a dexterous-manipulation competition and benchmark held at IROS 2026, organized by 13 co-organizers with the Nippon Origami Association as judge and task curator -> pretty cool to have them on board imho.

The evaluation consists of single task: a traditional Japanese paper airplane, in exactly six folds, from a 15×15 cm sheet of ≥60 gsm paper, on a competition-supplied standardized rig (bimanual arms + Sharpa Hands), both remotely and on-site.
Teams bring policies, not hardware.

An "Origami Grand Master" declares pass/fail on crease accuracy, structural fidelity, symmetry and paper integrity.
Among passes, faster folds rank higher, with a 10-minute-per-attempt ceiling and flight explicitly irrelevant to the score.
-> I find it interesting how they chose to evaluate the task. Quality is a binary pass or fail, therefore speed becomes the only thing graded here.
Speed is currently the bottleneck in dexterous manipulation though, so this choice makes sense.
I wonder whether there could be finer ways to grade the qaulity of such a creative task though.

When it comes to data, registered teams get 500+ teleoperation episodes (six camera streams, 65-D joint state/action, 10-fingertip 6-axis tactile), an NVIDIA Isaac Sim environment with thin-shell paper physics (plastic creasing + fold memory), digital twins of every partner hand, and a remote eval lab (upload a policy, queue an eval, get scored).

Still, I think it is a great dexterity benchmark the field badly needs, it supplies the hardware, an outside human judges, and the pass criterion is externally defined -> all three degrees of freedom are checked!
Neutral measurement layer, here we go!

The task is engineered to be un-gameable and to isolate pure dexterity.
A known figure, exactly six folds, judged on creases, with flight explicitly declared irrelevant (the latter makes sense to me).
Therefore, this underlines the goal to focus on dexterity, not task-selection or other strategies.

I really like origami as an ideal controlled dexterity task: deterministic goal, deformable medium, sequential, bimanual, precision-bound.
I am just not quite satisfied again by the binary pass or fail, I think quality of execution could be finely graded!
But again, I understand this is not the goal yet.

Also interesting to see the Sharpa Hands as de facto standard for everyone.
Total land-grab that anoints Sharpa as the reference dexterous hand, also featured in Google's Gemini Robotics 2.
By providing the hardware, the benchmark measures software while quietly making "good on Sharpa" the definition of good, and Sharpa gets real world data and feedback as a bonus.
That's smart, the data flywheel starts spinning.

The provided dataset is the richest tactile-manipulation corpus I have seen yet: 10-fingertip 6-axis tactile, plus plastic creasing and fold memory.
500+ teleop episodes with six camera streams, 65-D joint state/action, and ten fingertip 6-axis tactile sensors.
The force/tactile channel are parts of the the benchmark's data, this is the first time I see this.

Credits where it's due: organizers include @gao_young, @Noriaki_Hirose, @bgxc, @chris_j_paxton, @DJiafei, @micoolcho, @michaelyuancb, @hq_fang, and others.

[Yoonseok Yang @yoonseok_yang]

A lot of people have been asking what actually happened during my 1:1 with @JeffDean.

I didn’t expect my original post below to blow up like this, so here’s the full story of what happened:

[Zhening Huang @ZheningHuang]

Introducing LiteReality-Agent 🏠🤖✨

An agentic system that turns room scans into realistic, interactable 3D scenes.

No tool currently exists that can turn an entire room scan into an interactable 3D scene, so we built one! 🚀

Scan your room with the app we built, then let LiteReality-Agent turn it into a cool digital copy with articulated assets. 🛋️✨

Have fun! 🚀

🌐 Project page: https://t.co/8SpAtf1YAX
💻 Code: https://t.co/mxBxGO59C6
📱 App: https://t.co/yqcsJEFVPL
📝 Blog: https://t.co/8ffR3PmTAG

[國島佑介 | AIと宇宙の安全を考える人 @YusukeGuodao]

FriStiとAcompanyさんでClaude Code Security勉強会開催してます！

[Robots Digest 🤖 @robotsdigest]

Origami might be one of the hardest robotics benchmarks ever.

Not because it's complicated.
Because paper punishes everything—timing, force control, bimanual coordination, and tiny contact errors.
Sharpa's new Origami Challenge might become the ImageNet moment for visuo-tactile manipulation.

[Robots Digest 🤖 @robotsdigest]

Most world models predict what the world will look like after an action.

GeniWorld predicts the future from visual actions instead.

Instead of relying on robot-specific control commands, it learns from how objects move and interact in pixels, making the same world model transferable across different robots and embodiments.

[Chris Yoo @pentestduck]

Cloud inference will never be fast enough for robotics?

Here's MolmoAct2 VLA model running over cloud on an SO-101 arm.

Smooth motions despite the DC being 1000km away.

[Masato Kobayashi @るっと🐺 @MeRTcooking]

ロボット分野のトップカンファレンスの１つである #IROS2026 に採択しました🙌
バイラテラル制御に基づく模倣学習をマルチモーダルな階層型へ拡張しました！画像のキーフレームなメモリやサブタスクの進捗度合い等を見ながらHigh policyとLow policyが協調しながら位置と力制御を駆使して動作します！

[ClaudeDevs @ClaudeDevs]

New in Claude Code: your sessions can now message each other.

Instead of having to re-explain yourself in another session, you can now tell Claude to do it. It sends a summary (not your history or files), and the other session picks it up mid-task.

[CyberRobo @CyberRobooo]

Human priors hold way too many untapped data treasures…

Alibaba’s Qwen Team just released Ego2Robot.

A pipeline that turns large-scale egocentric human manipulation videos into robot training data.

From ~1,940 hours of ego footage, it produces 18,561 hours of synthetic robot data across 15 morphologies through action retargeting, robot-arm visual synthesis, and multi-level quality filtering.

When co-trained 1:1 with real robot data, a VLA model gains clear OOD robustness on visual appearance, scene layout, and task semantics. Synthetic data alone falls short--the hybrid mix is what works.

[Léo @LeoKharon]

NEW RESEARCH: You can now create a new robot optimized for any given task!

I love this new project by @haqhuy, @SongShuran, and others.

Called "Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-design", it generates a robot's physical design and its controller together from a task spec.

DEFINITIONS:
- Reward function: A scoring rule that assigns a number to how well a behavior achieves the task. Here, it is the objective the generated design is pushed to maximize (e.g., track the target motion with low error).
- Tokenizing: dividing continuous or structured data (a robot's links, joints, motor specs, states, actions) into a discrete vocabulary of symbols a transformer can process, the same step that turned pixels and audio into "language" for these models.
- Diffusion transformer (DiT): A transformer trained to turn random noise into structured output through iterative denoising. Here, it generates robot bodies and trajectories instead of images.
- MuJoCo: The standard fast physics simulator for robotics research (DeepMind-maintained). The Menagerie is its curated zoo of ready-to-use robot models.
- CMA-ES: Covariance Matrix Adaptation Evolution Strategy, the workhorse black-box optimizer: it evolves a population of candidate designs, keeps the best, and needs thousands of simulator rollouts.
- Bimanual multi-trajectory optimization: Finding one design/controller that performs well across several target motions for a two-armed robot at once, harder than optimizing for a single arm and a single motion.
- BERT/MAE masked-modeling trick: Train one model to fill in whatever parts of the input you hide (words for BERT, image patches for MAE); at inference, choosing what to mask chooses the task, so masking the body makes it a designer and masking the actions makes it a controller.

In practice, you give it a target end-effector motion and a reward function, and it outputs a complete embodiment (link, joint, motor, and inertial property), as well as a controller to drive it.

It works by tokenizing both the body (links/joints/motors) and the dynamics (states/actions) into a compact scheme called RoboTokens, training a diffusion transformer (DiT) over them.
The same model predicts dynamics using those predictions ("Dynamics Self-Guidance") to push generated designs toward higher reward at inference time.
Masking different token types (using the BERT/MAE masked-modeling trick) lets the one model do three jobs: generate an embodiment, control an arbitrary embodiment, or design one conditioned on a motion.

It is trained on 11 robots from the MuJoCo Menagerie (0.65 kg hand to 67.5 kg quadruped, 6–35 joints), and validated in sim and on a physical ALOHA doing cloth flinging.

I like the fact that this approach inverts the entire recent robotics ideas: designing a policy for a fixed robot -> designing the robot for a fixed task.
Every other approach assumes the body is given and learns a controller.
Transformer Transformer takes the task (target motion + reward), then generates the body and controller jointly.

In practice, it is a ~180× speedup over the standard optimizer at equal-or-better quality.
It reaches "CMA-ES-level quality in seconds" and finishes bimanual multi-trajectory optimization in <1 min vs CMA-ES's 3+ hours.

Surprisingly, this approach was validated on real hardware!
The generated design was measurably smoother.
On a physical ALOHA flinging cloth (a deformable, dynamic task), the optimized design cut tracking error 73% and peak joint speed 30% versus the original, with fewer joint-speed peaks.
Therefore: real world results, not a sim-only benchmark -> that is worth underlining nowadays!

Also worth mentioning: this is the lab behind UMI and Handroid, that I mentioned here previously!
The team seems extremely creative, i love these out-of-the-box approaches.

Enjoy watching the demo of robot optimization in 3D, data acquisition, then real-life testing: