Arnaud Fickinger

PhD Student at @berkeley_ai. Visiting Researcher at @facebookai.

[Arnaud Fickinger @arnaudfickinger]

@iamtrask @memdotai mem it

[Arnaud Fickinger @arnaudfickinger]

Find out more about Gromov-Wasserstein imitation learning at our poster Wed 4/27 10:30am PT at @iclr_conf.
Poster: https://t.co/wE7f6KcYtP
Paper: https://t.co/Pkuyl15n8p
Code: https://t.co/0T1L7zGTQw

[Arnaud Fickinger @arnaudfickinger]

We provably achieve cross-domain transfer in non-trivial continuous control domain by minimizing the Gromov-Wasserstein distance with deep RL.
Paper: https://t.co/qELtAYlG6t
Site: https://t.co/PHsy7HZADU
Joint work with @CohenSamuel13 Stuart Russell @brandondamos

A thread:

[Noam Brown @polynoamial]

We'll present our #NeurIPS2021 paper on scalable online planning via RL fine-tuning, which achieves SOTA performance in self-play Hanabi, at the poster session on *Thursday* Dec 9th 8:30-10am PT. I hope you'll join us! https://t.co/DgPJGwkk3k

[Arnaud Fickinger @arnaudfickinger]

In the Atari Ms. Pacman environment, RL Fine-Tuning outperforms MCTS in terms of sample efficiency and wall-clock time efficiency.

[Arnaud Fickinger @arnaudfickinger]

In our NeurIPS 2021 paper, we replace tabular search (MCTS, CFR, SPARTA) with inference-time neural fine-tuning. This results in a scalable search method that achieves a new SOTA in the Hanabi benchmark.

https://t.co/VX6kELPSxP

w/ @HengyuanH, @brandondamos, @polynoamial

🧵->

[Arnaud Fickinger @arnaudfickinger]

Hanabi is a stochastic, partially observable and cooperative game that reaches the limits of tabular search. Unlike tabular search, RL Fine-Tuning scales well with Hanabi and achieves a new SOTA.

[Arnaud Fickinger @arnaudfickinger]

@IevgenRedko @CohenSamuel13 @brandondamos It is indeed a direction that is worth exploring. I have integrated COOT into the code and am testing it on the benchmark. Would you be interested in having a call and chatting about it further sometime? If yes feel free to send me an email or pm.

[Arnaud Fickinger @arnaudfickinger]

We provably achieve cross-domain transfer in non-trivial continuous control domain by minimizing the Gromov-Wasserstein distance with deep RL.
Paper: https://t.co/qELtAYlG6t
Site: https://t.co/PHsy7HZADU
Joint work with @CohenSamuel13 Stuart Russell @brandondamos

A thread:

[Pierre Vandergheynst @trekkinglemon]

Yeah this is a paper I was very much expecting, beautiful!

[Arnaud Fickinger @arnaudfickinger]

@IevgenRedko @CohenSamuel13 @brandondamos Thanks for pointing out this nice work! We have added a reference to it and will submit an updated version to arxiv.

[Arnaud Fickinger @arnaudfickinger]

@SketcherRami @CohenSamuel13 @brandondamos 3. It is a reasonable follow-up work to consider.

[Arnaud Fickinger @arnaudfickinger]

@SketcherRami @CohenSamuel13 @brandondamos 2. Theorem 1 gives a sufficient condition for positive transfer: A's reward, transition and initial state distribution can be recovered from E's via isometric transformation of the state-action space. Our experiments show that is it not a necessary condition.

[Arnaud Fickinger @arnaudfickinger]

@pcastr @CohenSamuel13 @brandondamos Nice work on making bisimulation more robust! Although bisimulation-type metrics are indeed well suited metrics, they require more samples from the expert domain than directly computing the euclidean distance with a single trajectory so we need to explore the tradeoffs.

[Arnaud Fickinger @arnaudfickinger]

@pcastr @CohenSamuel13 @brandondamos Thanks for these great references! The distances defined in these papers depend on the reward while we don't assume access to it in our setting, but we are indeed considering better suited inner metrics in future works.

[Arnaud Fickinger @arnaudfickinger]

@pcastr @CohenSamuel13 @brandondamos Theorem 1 sheds light on how the expert's and agent's state-action metrics should relate to each other to have theoretical guarantees. In practice, the euclidean distance works well in all our experiments.

[Arnaud Fickinger @arnaudfickinger]

Our theory formally characterizes the scenarios where GWIL preserves optimality, revealing its possibilities and limitations.

[Arnaud Fickinger @arnaudfickinger]

A walker applying GWIL with a single cheetah demonstration and without any external reward can learn to move backward and forward by literally imitating a cheetah!