Those were the days, my good friend
We wrote gradients down by hand
They weren't too complex
Our losses were convex
You’re too young, you won't understand
The conference of NeurIPS has come
Our tests and models just begun
The researchers and nerds
Have come from all worlds
To show off their work, one by one.
@alienelf@fhuszar Leaders who are most effective
Take feedback and are self-reflective
But in Jade's DM
She witnessed these men
Who are instead rather defensive
About ML, sadly, he hasn’t a clue
Oh, how I hate this man, reviewer two
The review had no merit
I don’t think he read it
I wish he would get sick or fall into poo
There’s no way around it, I have to hereby
Declare that my ResNet is pre-AGI
It costs a Ferrari
To train on Atari
It can't put my socks on or pour me a chai
I talked to my good friends at Microsoft
We needed some money to refurb the loft
Satya said "sure,
But please use Azure"
A billion dollars will keep us aloft
Thank you for joining us at ICML
This year we're promising a whole new level
Of deep nets and GANs
Muslim travel bans
And more west-coast tech bros than ever
@fchollet PyTorch autograd was inspired by Chainer.
Keras was copied on Torch7 (transcribed from Lua to Python).
Torch7 was very much inspired by Lush (transcribed from Lisp to Lua).
But every time, some new twists are added.
How do I order a tarte à citron?
“Lemon tart” responds the Translatatron
End-to-end speech-to-speech
Is not out of reach
If you have data and wealthy patrons
A common misconception is that the risk of overfitting increases with the number of parameters in the model. In reality, a single parameter suffices to fit most datasets: https://t.co/4eOGBIyZl9
Implementation available at: https://t.co/xKikc2m0Yf
Look at this stuff: it's a StyleGAN
the output looks just like a floor plan
A flat without bedroom
With seven feet headroom
And living room shaped like a saucepan
https://t.co/MaQfrjholu
He erects no paywall, he charges no fee
Andrej is sharing his wisdom for free
The number you all want,
The Karpathy constant:
Ten to the minus fourth power by three
https://t.co/WidTJgx4CZ
New blog post: "A Recipe for Training Neural Networks" https://t.co/5lBy4J77aS a collection of attempted advice for training neural nets with a focus on how to structure that process over time
It took a researcher with some gumption
to revisit deep energy functions.
Try to normalize
and you'll bleed out your eyes;
It's intractable (by construction).
Progress towards stable and scalable training of energy-based models:
💻Blog: https://t.co/gMo8g5sctT
📝Paper: https://t.co/F9UlHf8JzI
🔤Code: https://t.co/WGIOX4OKxZ
@RandomlyWalking@MLimericks His thesis was done in 2013
His online trail is not so clean
Though his account has no name
His e-mail’s not the same
And the e-mail is easy to glean
On this feed you will continue to see
Limericks no longer written by me
But a person I tasked
For I’ve been unmasked
By the GAN boss. Good bye, C’est la vie
I try to be funny but mostly just troll
ImageNet’s too hard I stick to Swiss roll
Indeed, I’m Ferenc
I end this pretense
It’s time to put somebody else in control
Remember vacuous bounds do not spark joy
Nobody would agree more than Dan Roy
He doesn’t get sarcasm
Dislikes the huge chasm
Between the Bayes risk and bounds you employ
https://t.co/ookVfERVqw
Will people stop citing generalization bounds that are vacuous!!! There is no such thing as a tighter vacuous generalization bound. If it is vacuous then the value is 1!!
Pearl told us ML was curve fitting
This lead to lots of folks admitting
They may have to buy
Hashtag-Bookofwhy
Best selling book for the unwitting
https://t.co/0PAmjqvbNL
@PHuenermund Most economists understand that curve fitting is not causal inference, so they excuse themselves with: "we assume ignorability" which makes them feel less guilty. In contrast, Most machine-learning researchers do not see a reason to apologize. #Bookofwhy#econbookclub