"Double-dipping" - generating a hypothesis based on your data, and then testing the hypothesis on that same data - is dangerous. To see this, let's take data with no signal at all ... 1/
Have you been thinking about “what is being transferred in transfer learning?” and what parts of the network are in charge of that? We have some answers for you! https://t.co/74I8ec1Tix with @bneyshabur and Chiyuan Zhang
The Bias-Variance Trade-Off & "DOUBLE DESCENT" 🧵
Remember the bias-variance trade-off? It says that models perform well for an "intermediate level of flexibility". You've seen the picture of the U-shape test error curve.
We try to hit the "sweet spot" of flexibility.
1/🧵
Videos for my Fall 2019 course "Deep Learning for Computer Vision" are now on YouTube!
This is an evolution of @cs231n that I used to teach at Stanford:
- All content refreshed
- New topics: Transformers, Video, 3D, etc
- HW in @PyTorch + @GoogleColab
https://t.co/6nqZKTpmxv
All videos and slides for our Introduction to Deep Learning Lecture (I2DL) with @lealtaixe are now public!
Covering: ML basics, Neural Networks, Optimization, Training Processes, RNNs..
Videos: https://t.co/dQFQTiR5iP
Slides: https://t.co/GJ2Jn2GswA
#deeplearning@TU_Muenchen
Several people asked me for advice on how to best set parameters for histogram thresholding, so I made a little interactive tool that lets you explore the parameter space. Playing with this was more fun than I expected! https://t.co/H5uBkmi3Bp
Q. Can we find winning lottery tickets, or sparse trainable deep networks at initialization without ever looking at data?
A. Yes, by conserving "Synaptic Flow" via our new SynFlow algorithm.
co-led with Daniel Kunin
& @dyamins, @SuryaGanguli
paper: https://t.co/642Clw00me
1/
Introducing SimCLR: a Simple framework for Contrastive Learning of Representations. SimCLR advances previous SOTA in self-supervised and semi-supervised learning on ImageNet by 7-10% (see next).
https://t.co/X5CXud0VwL
Joint work with @skornblith@mo_norouzi@geoffreyhinton.