Vinal Gadhiya

Epoching...

[Vinal Gadhiya @vinalgadhiya]

@akhil_bagaria Thanks for sharing! will definitely give it a read!

[Vinal Gadhiya @vinalgadhiya]

Been going through some of @RichardSSutton 's older work, particularly the 1999 options paper with Doina Precup and Satinder Singh, and some of the earlier work on TD models and planning at multiple timescales in RL.

It's incredible how much of the work done in the 80s and 90s is still relevant to the problems we're trying to solve today.

One thing I keep coming back to is the idea of temporal abstraction. Humans don't think about every individual action when solving a problem. We think in terms of goals, subgoals, and things we already know how to do. And somehow we learn these abstractions through experience.

The options framework gives us a way to think about this mathematically, but i find the question of where these abstractions should come from even more interesting.

What even makes a skill worth learning? is it something that helps achieve a goal, something reusable across different tasks, or something that makes the world easier to understand and predict?

And if we think about learning world models at different timescales, as Sutton was already exploring back in the 90s, maybe the question isn't just how an agent should learn to predict the world, but also what parts of the world are worth predicting and reasoning about at different levels of abstraction.

should these abstractions be driven by rewards, or should they emerge from the structure of the environment and the agent's own experience?

I keep coming back to these questions when thinking about long-horizon learning and planning. Feels like there is still so much to understand here.

Almost three decades later, so many of these ideas are still at the center of what we're working on.

Would love to know what @RichardSSutton thinks about where hierarchical world models and self-supervised skill discovery fit into all this. can an agent discover meaningful temporal abstractions just by interacting with the world, without needing a particular task or externally defined reward to tell it what's useful?

[Vinal Gadhiya @vinalgadhiya]

Thanks for sharing these! Had a look at both papers, really interesting work, especially the adaptive option durations in SOL and the idea of combining reward functions to learn different behaviors in HBS. quite close to what i've been looking into and working on recently. will definitely dig deeper into both!

[Vinal Gadhiya @vinalgadhiya]

@TheHumanoidHub @itsajchan Amazing event! thanks to everyone who made it happen!

[Vinal Gadhiya @vinalgadhiya]

I've looked into H-JEPA. really interesting work, and some of the ideas are definitely helpful! Though as far as i understand, it uses fixed temporal strides at each level. i'm more curious about how an agent could discover meaningful temporal abstractions on its own, including what skills/options to learn and when they should terminate.

[Vinal Gadhiya @vinalgadhiya]

@danellisona what's the secret?

[Vinal Gadhiya @vinalgadhiya]

@KrishnaBuildsAI Wait until someone drops the “attention is all you need” equivalent for robotics and continual learning from interaction.

[Vinal Gadhiya @vinalgadhiya]

@RichardSSutton Thanks for pointing me to this! Also came across Oak Lab, really interesting research direction and quite close to what I've been thinking about. will dig deeper into the work!

[Vinal Gadhiya @vinalgadhiya]

@EbiThinks A friendly game

[Vinal Gadhiya @vinalgadhiya]

@aliattar Great meeting you guys at the event! had so much fun talking to and interacting with Lumi. such a cool and friendly robot!♥️

[Vinal Gadhiya @vinalgadhiya]

@heyartzy @TheHumanoidHub @itsajchan @nvidia Yeah! i'm actually more excited about running policies and world models on the edge while keeping simulation on my GPU server. should be fun to experiment with!

[Vinal Gadhiya @vinalgadhiya]

What an incredible two days at the Humanoid Hub Conference!! 🤖🔥

Huge thanks to @TheHumanoidHub , @itsajchan , and the team for an amazing event!

And to top it all off, I somehow went home with an NVIDIA Jetson AGX Thor!! Huge thanks to @nvidia for this beast! 🎉

Really enjoyed hearing from researchers and founders across 1X, Skild AI, Google DeepMind, Dyna Robotics, OpenMind, Lightberry, Auki Labs, Prometheus, and more. So many interesting talks, different perspectives on where robotics and physical AI are heading, and some super cool live demos!

Also got to meet some amazing people and have great conversations about what they're working on. Everything from the speakers and demos to the networking was so well organized!

I've been working on RL and world models for robotics, mostly in simulation, focusing on sample efficiency and long-horizon planning. I've been wanting to experiment with deploying learned policies and world models on edge hardware, and now I actually have a Thor to try them on!

Couldn't have asked for a better end to the week! 🎉

[Vinal Gadhiya @vinalgadhiya]

@TheHumanoidHub @itsajchan @nvidia Thank you!! Had a great time! Looking forward to more events and will definitely share what I end up building with the Thor!

[gaurav @gaxrav]

adulting is basically arriving at the same truths as your father, but from first principles.

[Vinal Gadhiya @vinalgadhiya]

Trying to increase my attention span but my brain keeps opening new tabs… internally.
What’s actually helped you focus better?

[kache @yacineMTB]

you can outsource your thinking
but you cannot outsource your understanding