Jonathan Clark

Research Scientist @ Gemini Multilinguality. Learning, Languages, Evals, C++. Previously MT@Microsoft and CMU. Opinions are my own.

[Dipanjan Das @dipanjand]

I am hiring for 2 positions focused on Gemini and information tasks in Mountain View. One role is particularly focused on pushing the frontier of factuality of LLMs.

We will consider applications for 1 week and then interview:

https://t.co/mzk8uiFcyY

https://t.co/t5y4XA0vnZ

[dynomight @dynomight7]

DumPy: Like NumPy except it's OK if you're dum

[Dipanjan Das @dipanjand]

Today, we have released a benchmark and a corresponding leaderboard titled "FACTS Grounding" in collaboration with @kaggle. 1/

[Markus Freitag @markuseful]

Catch our Google Translate Research team at #EMNLP #WMT24! The team will present 9 papers on step-by-step decoding, mitigating metric bias within MBR decoding (+ MBR dataset release), improved human data collection and automatic metrics (MetricX: winner of WMT Metrics Task).

[Jason Riesa @jasonriesa]

🚀 Join the Gemini Multilinguality team @GoogleDeepMind 🌐 We’re looking for researchers passionate about making LLMs helpful for all. Dramatically improve model quality, coverage, and cultural relevance across hundreds of languages. #NLProc #MultilingualAI #i18n #LLMs https://t.co/RzwiRM5zWr

[Google Canada @googlecanada]

Exciting news! As of today, the Inuit language of Inuktut will be available on Google Translate - marking the first Canadian Indigenous language on the platform. Tunngasugit! | ᑐᙵᓱᒋᑦ (Welcome!) 🎉
Huge thanks to @ITK_CanadaInuit for their invaluable guidance and collaboration.
Learn more on our blog: https://t.co/WPvtG5stmL

[iseeaswell꩜bʂky @iseeaswell]

Excited to announce that 110 languages got added to Google Translate today! Time for context on these languages, especially the communities who helped a lot over the past few years, including Cantonese, NKo, and Faroese volunteers. Also, a 110-language youtube playlist. 🧵

[Jeff Dean @JeffDean]

As part of @Google's 1,000 Languages Initiative, a commitment to support the 1,000 most spoken languages, & w/help of our PaLM 2 LLM, we're adding support for 110 new languages (spoken by 614M people) to Google Translate (now supporting 243 languages). 🎉
https://t.co/DRiw03V0wu

[Shruti Rijhwani @shrutirij]

I will be at NAACL in-person, and co-organizing the AmericasNLP workshop there!

#NLProc #NAACL2024

[Jing Yu Koh @kohjingyu]

Absolutely unhinged.

When @jasonbaldridge started this in 2021 he would enthusiastically show us weird new images that he took. I thought it was just some weird phase that would fizzle out, but I'm very happy to be wrong, and that it resulted in such a high quality dataset!

[Markus Freitag @markuseful]

New paper alert!

Designing reliable human evaluation is both crucial and difficult. Human raters can exhibit different behaviors when rating NLG outputs. These differences are not generally due to a rater performing the task incorrectly, but rather due to differences in harshness or leniency between raters: a Minor error to one rater may be a Major error to another. Consequently, decisions around which raters rate which items can alter the final system ranking.

In our new paper, we analyse the impact of rater assignment on the final system ranking and show how you can design a replicable, reliable human evaluation by assigning the right raters to the right items.

Take a look: https://t.co/2iOf5AaPvd

[Graham Neubig @gneubig]

ACL has removed the anonymity period.

This means that ACL submissions can be posted and discussed online at any time, although extensive PR is discouraged.

https://t.co/F7xVdCEoyY

[John Wieting @johnwieting2]

Today at #NeurIPS2023, If you want to lean more about:

1. Robustness of detectors and watermarks to paraphrase attacks (spoiler alert: needs improvement).

2. An alternative detection approach using simple retrieval methods.

and ...

[Dan Deutsch @_danieldeutsch]

Excited to receive an Outstanding Paper award for this work at @emnlpmeeting! Thanks to my co-authors George Foster and @markuseful! Updated version available here: https://t.co/XINveU1LvG

[John Wieting @johnwieting2]

Our work on cross-lingual and multilingual attribution will be presented at #EMNLP2023 in Singapore!

We have also released our dataset of ~10k 3-way annotations over 5 typologically diverse languages.

Arxiv: https://t.co/dAdN4wRwnS
Dataset: https://t.co/RsTYZOi7GD

[Benjamin Muller @ben_mlr]

Excited to be presenting our work on
**Evaluating and Modeling Attribution for Cross-Lingual Question Answering** at #EMNLP2023 in Singapore.

Updated Paper: https://t.co/YPxQgDVAd2
We're also releasing the XOR-AttriQA dataset: https://t.co/enUl21jB3h

🧵

[Dipanjan Das @dipanjand]

Excited to announce the First Conference on Language Modeling, to be held in approximately a year from now.

Please let us know if you are interested or have any feedback on the conference: https://t.co/YbDsZO9goT

[Sasha Rush @srush_nlp]

Introducing COLM (https://t.co/7T42bAAQa4) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models.

Submissions: March 15 (it's pronounced "collum" 🕊️)

[iseeaswell꩜bʂky @iseeaswell]

Have you ever wanted a LangID model that works on 1500+ languages? check out FUN-LangID: https://t.co/Uy5F223VW6 !

[Sneha Kudugunta @snehaark]

Excited to announce MADLAD-400 - a 2.8T token web-domain dataset that covers 419 languages(!).

Arxiv: https://t.co/Y48Bsw952P
Github: https://t.co/ANNVWvfAk2  1/n