Yongwei Che

Currently @ Radix Trading, Princeton 2024

[Yongwei Che @CheYongwei]

I'll be presenting this work at ICLR 2025 in Singapore! Stop by to learn more :)

- Poster: Fri 3pm-5:30pm Hall 3 + Hall 2B #328

#ICLR2025

[Yongwei Che @CheYongwei]

1/ When using multimodal models, we often want to infer over modalities unseen together in training. Prior works typically address this by learning a shared contrastive embedding space between modality pairs, but is this approach principled? We prove that it is! (under certain assumptions)

Paper+Code: https://t.co/d2bOm79ie7

[Yongwei Che @CheYongwei]

3/ We apply our results to language-conditioned reinforcement learning, showing better goal-reaching when prompted with ambiguous language goals! Direct evaluation navigates to the "mean embedding", while our LogSumExp takes the shortest path.

[Yongwei Che @CheYongwei]

1/ When using multimodal models, we often want to infer over modalities unseen together in training. Prior works typically address this by learning a shared contrastive embedding space between modality pairs, but is this approach principled? We prove that it is! (under certain assumptions)

Paper+Code: https://t.co/d2bOm79ie7

[Yongwei Che @CheYongwei]

2/ We also provide a Monte Carlo algorithm (coined LogSumExp) for when these assumptions aren't met, allowing us to infer across multimodal pre-trained models that weren't trained together (such as CLIP and CLAP)