Learning from human videos often requires restrictive, carefully choreographed human motions.
We propose ✨3PoinTr✨: a scalable way to pretrain from casual human videos. It bridges the embodiment gap by learning 3D scene evolution, enabling learning from natural human motions.
How can we get robot hands to “hear” slip and contact through microphones, and react to them?
We’re excited to share VibeAct, an approach that uses piezoelectric microphones embedded in robot fingertips to estimate contact and slip, then learns reactive policies from this tactile feedback!
https://t.co/7jMLZNpYzp
Excited to share the first paper of my PhD!
If you’ve ever tried to control a VLA via natural language, you know it rarely does what it is told. 🗣️ We introduce a multi-stage pipeline for training a Language Feedback Policy (LFP) to steer a VLA in-the-loop.
Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation!
Text-to-image (T2I) models learn rich representations of the spatial world.
How do we build on this prior for high-quality depth generation?
https://t.co/uJjGHNiDBu
🧵 [1/6]
Excited to share SoftAct, a framework for retargeting human manipulation demos to soft robot hands using explicit contact force reasoning! How do you transfer human skill to a hand that looks and moves nothing like yours🐙🖐️? It turns out VR environments can let us capture privileged force interaction demonstrations to help. 🧵1/7
I’m so tired of writing rebuttals to this kind of “lack of novelty” review: “This paper trivially combines A, B, and C, so the algorithmic novelty is limited.”
Technically, most (if not all) robotics papers are convex combinations of existing ideas.
I still deeply appreciate A+B+C papers—especially when they deliver:
- New capabilities: the “trivial combination” unlocks behaviors we simply couldn’t achieve before
- Sensible & organic design: A+B+C is clearly the right composition—not some arbitrary A′+B+C′
- Nontrivial interactions: careful analysis of the dynamics, coupling, or failure modes between A, B, C
- Rehabilitating old ideas: A was dismissed for years, but paired with modern B/C, it suddenly works—and teaches us why
- System-level & "interface" insight: the contribution is not any single piece, but how the pieces talk to each other
- Scaling laws or regimes: identifying when/why A+B+C works (and when it doesn’t)
- Engineering clarity: making something actually work robustly in the real world is not “trivial”
- New problem formulations: sometimes the real novelty is in the reformulation—only under this view does A+B+C make sense.
Maybe worth keeping these in mind when reviewing the next A+B+C paper : )
📑Paper: https://t.co/pdDzg8gfsB
🌐Website: https://t.co/l4bBBbMBOA
@CMU_Robotics@SCSatCMU
Thank you to collaborators @BDuisterhof and @jeff_ichnowski !! This work was also supported by the NSF GRFP.
Learning from human videos often requires restrictive, carefully choreographed human motions.
We propose ✨3PoinTr✨: a scalable way to pretrain from casual human videos. It bridges the embodiment gap by learning 3D scene evolution, enabling learning from natural human motions.
By enabling scalable 3D pretraining from casual videos, 3PoinTr takes a step towards future work in making effective use of internet-scale, in-the-wild human interaction data for learning generalist robot policies.