Pushpak Agrawal

it's all about weights | Research@CMU

[Pushpak Agrawal @pushpak_agr]

@DozenDucc @yiding_song @theWaddleLabs got it. curious to know, is there a model in the pipeline for estimating the trajectory or it's just claude making prediction in the wild?

[Pushpak Agrawal @pushpak_agr]

@shenzhenfoundry where to purchase from?

[Pushpak Agrawal @pushpak_agr]

@irvinxyz obv none!

[Pushpak Agrawal @pushpak_agr]

@ai_bites poor output. models are failing when tested on egocentric video collected at scale where environment and collection strategy isn't controlled

[Pushpak Agrawal @pushpak_agr]

@highvis_supply what camera rig is this?

[Pushpak Agrawal @pushpak_agr]

@lukas_m_ziegler @perceptroninc interesting right?

[Pushpak Agrawal @pushpak_agr]

@sean_pixel isn't this action captioning at it's best? you can't call this annotation pipeline for egocentric videos.

[Pushpak Agrawal @pushpak_agr]

@bilawalsidhu was IMU data used? if so, how did you read the IMU data from the Meta Ray-Bans?

[Pushpak Agrawal @pushpak_agr]

@kuberwastaken @snowmaker @ycombinator oh nice got it! I was standing there too, didn't see you haha

[Pushpak Agrawal @pushpak_agr]

world models.

[Pushpak Agrawal @pushpak_agr]

@AdemiAdeniji @pabbeel @svlevine @LerrelPinto @Ken_Goldberg @DrJimFan congratulations!

[Pushpak Agrawal @pushpak_agr]

things were stripped down to bare minimum. vocab size was 12 (0-9 chars, [CLS], "+") and context window was 4 with embedding dim as 32.

[Pushpak Agrawal @pushpak_agr]

wrote a simple CLIP architecture to understand contrastive learning and how dual encoder models organize latent space.
math equations (basic addition) were synthetically generated (pixel space) paired with their tokenized sequences (text space).
the architecture consists of a standard CNN image encoder and a lightweight self attention based transformer text encoder.
contrastive loss successfully aligns the manifolds without explicit labels.

[Pushpak Agrawal @pushpak_agr]

@amir @AravSrinivas @steph_palazzolo @HeeterMaria So what's the answer?

[Pushpak Agrawal @pushpak_agr]

Just blame the dataset and move on

[Pushpak Agrawal @pushpak_agr]

@PathikGhugare @AravSrinivas Exactly

[Pushpak Agrawal @pushpak_agr]

@tunguz vision transformer