@jeremyphoward@huggingface Single run with lr 2e-5 (except for distilroberta)
ELECTRA is the smallest 13.5m params
MiniLM - 33.3m
DeBERTa - 70.8m
DistilRoBERTa - 82.1m
those include embedding layer. Need to tweek logging to include num params w/o embeddings 🙂
@jeremyphoward@huggingface I was mostly using MiniLM for quick baseline and ELECRA-small if I need something really tiny.
Ran couple of experiments on SST2, here are the results. Another experiment with MRPC here https://t.co/aoHK96fcd2
"don't try to stop me or I'll nuke you" is not a threat you make from a position of strength. It comes from a position of extreme weakness, desperation even.
#Hungary is among the countries advocating against banning russia from the #SWIFT IPS. Previously, 🇭🇺 had banned 🇺🇦 from joining the @NATO cybercenter. We call on Viktor #Orbán to support 🇺🇦's choice for Euro-Atlantic integration and our inalienable right for self-determination.
Dear Scientific Community.
I would like to share this open letter from the Ukrainian academic community. Please, support me in sharing it further.
@ducha_aiki#StandWithUkraine
meant to post this after the official announcement of the results but I feel uncertain about tomorrow. I love doing DL but I cannot think of it now. Yet this one model is particularly important today as we need Ukrainian voice heard around the globe as we say HELP US STOP THE WAR
I want to thank @huggingface for organizing Robust Speech Recognition event. I had a great time fine-tuning cutting-edge ASR models. And I was happy to produce best performing models for chosen languages including Ukrainian, my mothers tongue https://t.co/R8pXl38fak
Thanks to @PatrickPlaten and @anton_lozhkov for hosting it @OVHcloud for providing compute @mozilla for Common Voice dataset @Kensho for their pyctcdecode library and everyone involved for making it such a pleasant event
Just incase you didn't know - that is happening and it is not funny at all.
Call your representatives. Go to the rally for Ukraine. Commenting on social media would not help #Ukraine.
I rarely write about politics on Twitter. What happens in Ukraine is a monstrosity. There are no nuances or “their version”, it’s black and white. The West is directly responsible for allowing the ugly growth of Putin’s regime by looking aside, taking their money and pretending
🚨It's time for another community event on July 7th-July 14th🚨
We've partnered up with @GoogleAI and @googlecloud to teach you how to use JAX/Flax for NLP & CV🚀
You define your project - we give you guidance, scripts & free TPU VMs 🤗
To participate:
https://t.co/VIgvLBbPaZ
:gift: :gift: Feedback Requested :gift: :gift:
Our @fastdotai community team would ♥️ feedback on our @paperswithcode Reproducibility Challenge @wandb report (1 day before the deadline 😬), reproducing the Reformer paper
📕: https://t.co/qJF9xuuUbw
Reply, DM... 1/2