DeepSPIN

Deep structured prediction in NLP. ERC project coordinated by @andre_t_martins. Instituto de Telecomunicações.

[DeepSPIN @deep_spin]

Our group is headed to #acl2019nlp! Come chat at our posters and presentations listed in the attached image. More info in thread 👇👇 [1/11]

[timorous bestie 😷 @vnfrombucharest]

Adaptively Sparse Transformers
@emnlp2019 +Gonçalo Correia, André Martins

α-entmax attention
α=1: softmax, α=2: sparsemax, continuous in between.
twist: we learn α for each head, w gradients! Some heads become dense, some sparse.

https://t.co/j5fnYbzytz
https://t.co/HRvUnIZP9S

[DeepSPIN @deep_spin]

(forgot to tag @pedrohenmartins!)

[DeepSPIN @deep_spin]

Our group is headed to #acl2019nlp! Come chat at our posters and presentations listed in the attached image. More info in thread 👇👇 [1/11]

[DeepSPIN @deep_spin]

Joint Learning of Named Entity Recognition and Entity Linking

Pedro Henrique Martins, Zita Marinho, André Martins

https://t.co/Y4frVMIWzM

Tue 30, SRW, Posters 2 10:30–12:10 [5/11]

[DeepSPIN @deep_spin]

Unbabel's Participation in the WMT'19 Translation Quality Estimation Shared Task

WMT QE s. t. winner!

Fábio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, António Góis, Amin Farajian, António Lopes, André Martins

https://t.co/u6ehU1rlEi

Fri 2, WMT APE 11:00–12:30 [11/11]

[DeepSPIN @deep_spin]

BERT-based Encoder-Decoder for Automatic Post-Editing

WMT APE shared task winner!

António Lopes, Amin Farajian, Gonçalo M Correia, Jonay Trénous, André Martins

https://t.co/yimSU3TXFk

Fri 2, WMT APE 11:00–12:30 [10/11]

[DeepSPIN @deep_spin]

Slides and plenty of useful resources at https://t.co/LMnsbygHQo

[DeepSPIN @deep_spin]

The Variational Inference Tutorial tour stopped by our lab! Thank you @wilkeraziz for a fantastic tutorial and a visit full of exciting discussion!

[DeepSPIN @deep_spin]

A nice write-up of the challenges of lemmatization by DeepSPINner Erick! Multilingual examples reveal different complexities hard to imagine if focusing on English.

[Erick Fonseca @Erick404]

My overview of state-of-the-art lemmatization https://t.co/RSE8hdl3rN

[DeepSPIN @deep_spin]

DeepSPIN talks at #emnlp2018 !

- Thu, 11:00AM, talk @ Blackbox NLP
Interpretable Structure Induction via Sparse Attention.
Peters/Niculae/Martins.

- Fri, 3:36PM, main conf talk @ ML(3B)
Towards Dynamic Computation Graphs via Sparse Latent Structure.
Niculae/Martins/Cardie.

[timorous bestie 😷 @vnfrombucharest]

Towards Dynamic Computation Graphs via Sparse Latent Structure: #emnlp2018 + André @clairecardie

- marginalize over structured latent vars w/ SparseMAP
- CG a function of discrete structure
- eg latent dependency TreeLSTM

pdf https://t.co/fKQ64doAey
code https://t.co/nodZwOi4Jh

[DeepSPIN @deep_spin]

"Structure Back in Play, Translation Wants More Context"

DeepSPINner André Martins writes on the @Unbabel R&D blog his notes from this year's #icml2018 and #ACL2018:

https://t.co/QPpZ6uGWaV

[DeepSPIN @deep_spin]

2) #BlackboxNLP Interpretable structure induction via sparse attention:
@bpopuw, @vnfrombucharest, André Martins

We discuss the interpretability angle of recent & ongoing work on sparse, structured and constrained attention: sparsemax, fusedmax, constrained sparsemax, SparseMAP

[DeepSPIN @deep_spin]

DeepSPIN will be at #emnlp2018!

1) SparseMAP for latent dynamic computation graphs; @vnfrombucharest, André Martins and @clairecardie;

2) Discussing the interpretability of sparse/structured attention; @bpopuw, @vnfrombucharest, Martins at #BlackboxNLP.

(preprints soon)

-->

[DeepSPIN @deep_spin]

1) "Towards dynamic computation graphs via sparse latent structure": @vnfrombucharest, André Martins, @clairecardie

- marginalize over structured latent variables with SparseMAP

- computation graph conditions globally on discrete structure

- e.g: latent dependency TreeLSTM