reeeemo

MSCS @CUBoulder, deep passion for visual intelligence/reinforcement learning and @Lions. Open for opportunities & joint collaborations.

[reeeemo @robobOxley]

@ja_rothschild Numerical! It used the "ikpy" library to derive the chain from the URDF file and iteratively drive the joints until the end effector's residual was minimal enough.

[reeeemo @robobOxley]

When I used SAM3 to build a dataset, short noun phrases like "puzzle" were fine but adding on to it (e.g., "puzzle pieces", "jigsaw puzzle pieces") ended up with worse segmentations than SAM2 with bounding boxes. Interesting to see how Astra catches those intricacies on unseen images, but I feel longer latency leaves it redundant in cases where inference latency needs to be minimized.

[reeeemo @robobOxley]

I agree. Under-actuated quadrotors seem largely constrained to a supporting role on long-horizon jobs; or to do short tasks on their own.

Cooperative multi-drone setups or a fully actuated / tilted multirotor can reduce the coupling between stability and reaction forces from the payload or environment (heavy wind, arm manipulation, etc.). Though power consumption and limited onboard energy still caps them as specialists.

[reeeemo @robobOxley]

@rboyd I found not training on the map forced me to learn separate approaches to unseen course generations, rather than training on the gate layout or an expert trajectory through it.

Despite not making it through VQ2, that constraint was an interesting challenge.

[reeeemo @robobOxley]

Just wrapped up the final polish on my updated Flightmare repo and released a full technical write-up of the autonomy stack I built for trajectory generation on autonomous quadrotors using IMU and vision estimates.

Repo + write-up:
https://t.co/CbNO5TVK34

[reeeemo @robobOxley]

Always happy to discuss RL, visual intelligence, or any form of autonomous robotics.

Throughout this competition I have absorbed a tremendous amount of knowledge from my own research, papers, and from quite a few of you who shared your work.

I'll continue growing in this domain and look forward to keeping building and sharing along the way!

[reeeemo @robobOxley]

Exceeded my goals for the Anduril #AIGrandPrix, finished VQ1 and here’s my best VQ2 run.

This has been an amazing opportunity to research and learn autonomous flight with a quadcopter. It's been a pleasure interacting and discussing techniques with many of you!

I will clean up and release my updated Flightmare repo (with the code I used to train the policy) as soon as possible.

[reeeemo @robobOxley]

Yeah I think the residual model of the drones perception was the side of my policy that was the weakest.

When it got extremely close to gates at numerous angles it caused quite a few crashes, my pose estimation model is weak detecting gates in those scenarios, and my policy had trouble dealing with the failure modes observed.

Extremely interesting information to explore later though for sure!

[reeeemo @robobOxley]

I also attempted to add next-gate to the observation. However as next gate is far more sparse of a detection than current gate it caused more issues during vision inference than intended.

This had to be converted to body frame to not directly compete with the current gate's relative world-coordinate position to the drone, it had the unintended side effect of causing premature rotations before the current gate was crossed.

I also attempted to use current gate in body rotation form, but the MLP can already learn the bilinear transformation (drone_rotation.T @ cur_gate). Thus this observation was dropped.

[reeeemo @robobOxley]

Been so busy tinkering with my RL model for VQ2 during the #AIGrandPrix I have not shared any results from VQ1 yet. Here is my best result:

[reeeemo @robobOxley]

Not perfect, but it it shows the successful transfer PPO policy from Flightmare -> VQ1 simulator. Significant updates since my last post:
- Domain randomization (30% randomization on steady state motor response, max/min angular velocity, etc.) following techniques from Ferede et al. (https://t.co/E7XJjBQpDH)
- Much larger environment variation to better match VQ1/VQ2 distributions (my previous post had close gates, new environment can have close or far in the same course). A larger spread of vertical and lateral movement in the gates is also observed.
- Reward function changes: dynamic floors/walls each episode to force precision at speed, plus dead-reckoning during training to match my VQ1/VQ2 setup.

[reeeemo @robobOxley]

This looks great so far! I am also using PPO. Was there anything that you particularly struggled with? There is a really good paper that covered DR (One Net to Rule them All) that helped me basically rebuild my sim so I did not have to match drone dynamics.

My largest issue so far has been configuring my vision stack / observation collection in the DCL sim to fit the distribution my policy sees.

[reeeemo @robobOxley]

@Brusco_RL Wow that is fast! Still trying to convert my RL model from Flightmare to the DCL sim. If you don't mind a question: it appears you used flightmare too from your previous posts, did you ever run into issues with the 6 rad multiplier in the C++ backend?

[reeeemo @robobOxley]

So the vision model detects the outer box and the inner box seperately? That would be a super interesting idea to explore, how did it work for you?

My only concern is the signal of the outer box being potentially weaker since you'd want to give value to the policy for being in the center of the detected keypoints.

[reeeemo @robobOxley]

Been working away at my model for the #AIGrandPrix for a little while now. Got the technical specs dialed in and after lots of experimenting and improvements, I have a qualifier 1 ready model :)

[reeeemo @robobOxley]

Note that the drone is sped up since rendering the scene with 2 cameras while computing physics slowed down the rendering time. Also, the policy in this video is running only on inferenced images taken from the camera to the vision model!