Fabio J Fehr

Postdoc ML @ Oxford TVG

[Fabio J Fehr @FabioFehr]

@NaceAI @ZacharyShinnick 🔥

[Fabio J Fehr @FabioFehr]

@DamienTeney Thanks @DamienTeney for the encouragement! The higher spearman's correlation shows the length is more adaptive to shorter and longer length summaries. In the Appendix (Figure 5 and Table 7) Curation Corpus and CNN/DM dataset we get improved OOD performance for longer summaries.

[Fabio J Fehr @FabioFehr]

I am so excited by our latest work on ArXiv!

TL;DR: We apply NVIB to pretrained Transformers which allows for an information theoretic post-training regularisation. With no weight updates we achieve improved performance in out-of-domain generalisation!
https://t.co/z7W8kelEJ9

[Fabio J Fehr @FabioFehr]

An overview of Large Language Models (LLMs) pitched in a very accessible way!

https://t.co/YObEq7Z0Tk

[Fabio J Fehr @FabioFehr]

I am really excited our short paper "Learning to Abstract with Nonparametric Variational Information Bottleneck" got accepted to the findings of EMNLP 2023!

A big thanks to my college Melika Behjati and supervisor James Henderson

[Tom McCoy @RTomMcCoy]

🤖🧠NEW PAPER🧠🤖

Bayesian models can learn rapidly. Neural networks can handle messy, naturalistic data. How can we combine these strengths?

Our answer: Use meta-learning to distill Bayesian priors into a neural network!

Paper: https://t.co/KwKEsSR8Vu

1/n

[François Fleuret @francoisfleuret]

"HyperMixer: An MLP-based Low Cost Alternative to Transformers" at ACL2023, with @_florianmai, @ArnaudPannatier, @FabioFehr, Haolin Chen, François Marelli, and @JamieBHenderson.

https://t.co/EHQEkyvzCX

@unige_en @Idiap_ch @EPFL_en

2/2

[Fabio J Fehr @FabioFehr]

To volunteer at a conference? - What are the pros and cons?

[James Henderson @JamieBHenderson]

I am excited to announce our @iclr_conf 2023 paper:
"A VAE for Transformers with Nonparametric Variational Information Bottleneck"
https://t.co/2LIMCaxMJF
We propose to model Transformer embeddings as nonparametric mixture distributions using Dirichlet processes. @FabioFehr

[Fabio J Fehr @FabioFehr]

I am very excited to announce my first paper towards my PhD has been accepted at @iclr_conf 2023 in Kigali, Rwanda!

"A VAE for Transformers with Nonparametric Variational Information Bottleneck"

@JamieBHenderson @Idiap_ch

https://t.co/C2bVHD199v
https://t.co/dTGIhm0ROf

[arXiv Daily @Arxiv_Daily]

A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck
https://t.co/E5f3z7Vih9
by James Henderson and @FabioFehr
#Autoencoder #Vector

[François Fleuret @francoisfleuret]

TL;DR: we propose an MLP-Mixer whose parameters are modulated by an MLP. It avoids the O(T^2), captures long-range dependencies attention-style, and is easier to meta-optimize!

With @_florianmai, @ArnaudPannatier, @fabiofehr, H. Chen, F. Marelli, and @JamieBHenderson.