Devesh Vyas

Robo | Math | Physics enthusiast. Trying to unstack challenging problems in robotics | Founders Inc

[Devesh Vyas @Dev272Dv]

ACT + Recovery & Correction

I’m thinking about it as two gates:

Recovery gate: “We’re drifting away from states ACT knows how to handle.”

→ trigger a recovery behaviour that brings the robot back toward a familiar state.

Correction gate: “We’re recoverable"

[Devesh Vyas @Dev272Dv]

@DavidStalmarck Is that pwm

[Devesh Vyas @Dev272Dv]

@signalgaining Can you breakdown end-to-end latency, control loop frequency, network round trip.

And is it running closed loop, open loop planning, or is it predicting chunks of actions

[Devesh Vyas @Dev272Dv]

@jukan05 Amd CPUs are pretty good. Have tried rocm software stack as well most of the kernels and code is portable.

Tho I ssh’d into cloud sesh with all drivers installed which i think most find difficult to hop around

[Devesh Vyas @Dev272Dv]

@0xGoldTooth What were they doing while hiring and 7-8 rounds of interviews is enough opportunity to weed our incompetent fella.

Should’ve sent a push back email saying what the heck were you folks assessing during interviews

[Devesh Vyas @Dev272Dv]

@cskvarma That's what I call for models small enough to be deployed on edge devices like a cheap raspberry pi and smart enough to solve general purpose tasks.

[Devesh Vyas @Dev272Dv]

@suhanthiru1925 I thought that was base frame or reference frame engraved on table nvm

[Devesh Vyas @Dev272Dv]

@erpriyankpatel Not only this but have pitched to VCs.

Got into couple of accelerators.

Been looking for a co-founder per se. Would have tapped my friends network but no one seems to be that interested in physical/embodied AI.

Says too difficult or don’t see it as a good biz model

[Devesh Vyas @Dev272Dv]

@erpriyankpatel A couple of tangents that I've been working on:

sub hundred million multi modal robot foundational models redwood neo inspo.

Soft object manipulation. Still a long way to go I think fusing tactile and vision.

Memory is long unsolved for long horizon tasks. Pi0.7 Mem was good

[Devesh Vyas @Dev272Dv]

@stoyanpstoyanov A couple of tangents that I've been working on:

sub hundred million multi modal robot foundational models redwood neo inspo.

Soft object manipulation. Still a long way to go I think fusing tactile and vision.

Memory is long unsolved for long horizon tasks. Pi0.7 Mem was good

[Devesh Vyas @Dev272Dv]

@Yue1650749 Wtf I read Albert Einstein as Albert Epstein

[Devesh Vyas @Dev272Dv]

@JamesMalsawm Hell no that’s AGI. That’s how humans work. It’s learning our patterns

[Devesh Vyas @Dev272Dv]

@asimovinc ball park bom. Sounds rad. Would deffo like to build a whole control policy and let it go wild in the wild. Btw does the repo has some baseline policies or any HF checkpoints that we could try off right off the bat

[Devesh Vyas @Dev272Dv]

@kgauravnm same feels like with sol and astra. Tho sonnet feels in pretty good shape. Tho needs bit more steering, clearer, deterministic kinda tasks.

[Devesh Vyas @Dev272Dv]

@DavidStalmarck @Gustav_Hugod @TomasHarmer @Julius48024685 In

[Devesh Vyas @Dev272Dv]

@anuragdotdev Could be out of distribution data, class imbalances, use stratified split, cross validation, data leakage a few prime suspects

[Devesh Vyas @Dev272Dv]

@viktor_vrp build a whole body control policy just to open boxes. As soon as a box arrives humanoid backflips in joy (pretty easy for it innit) and rips open all the stuff and shoves the stuff right in our faces and add a speech model which says what a waste of $$$

[Devesh Vyas @Dev272Dv]

@nuel_szy 240+420 =660
660-1

[Devesh Vyas @Dev272Dv]

@ameya_mjoshi @glowcone @Linyan_Fu @Apheth_DAlmeida I thought it should be other way round. Frame it as a normal arcade game slap a $2/5 fee for 10 minutes of play not sure how env resets would work but can be other source of rev for company

[Devesh Vyas @Dev272Dv]

@tarat_211 Work on motor backlash. Use tactile sensor on the gripper.

[Devesh Vyas @Dev272Dv]

@praneethsamine I was thinking it’d be a dense reward policy with rewards on contact, balance, angle, ball tracking, and other variables.

Also was thinking if somehow could make policy learn mgh and energy lost in friction and apply law of conservation of energy for vel, height tracking nvm