Depth First Learning

I like long walks down trees and impactful machine learning papers.

[Jess Lin @jesstyping]

I LOVE this talk by @ShriramKMurthi on how to design a curriculum to teach programming.

https://t.co/RO6Rosm8gg

It's tempting to jump to solution X. He pulls back to focus on *constraints*, just as you'd do in an eng design problem. 🧵

[Depth First Learning @DepthFirstLearn]

"I would love somebody to take away from this paper that datasets are situated. It's not just the perspectives of the creators but also the socio-technical processes like search engines and the time+place particulars that filter through in the act of creation."

[Depth First Learning @DepthFirstLearn]

Today, we have @cephaloponderer and @cinjoncin talking about Emily's paper from July 2020 --> Bringing The People Back In: Contesting Benchmark Machine Learning Datasets.

Check it out here: https://t.co/ALJDUQ4YSA

[Depth First Learning @DepthFirstLearn]

"There's a history of making these data sets. Well, what are the things that people bring to the table when they do that? If we can understand that, then we can see where the deficiencies are that could lead to things going forward that are just better approaches."

[Depth First Learning @DepthFirstLearn]

"... we invested a lot in monitoring and this was a pretty large scale effort. We're monitoring on the order of 10000 pieces of information a second. This was critical to our success because there's so many moving pieces and so many places where the optimization can go wrong."

[Depth First Learning @DepthFirstLearn]

This week, we have @Luke_Metz and @cinjoncin talking about Luke's 3 (now 4) year journey on learning optimizers.
Find the audio and transcript at https://t.co/FuKu2VK9ph. Some choice quotes in the thread 👇

[Depth First Learning @DepthFirstLearn]

"One difficulty in this work is just getting a better sense of what's going. People often monitor certain attributes in their models. But now there's not one model being trained. There's huge numbers of models being trained in very different settings. So throughout this work ..."

[Depth First Learning @DepthFirstLearn]

"... just want the model to be compact. We may want it to be interpretable or robust or fair. The message of this paper is that by optimizing for one property, we may be compromising others."

[Depth First Learning @DepthFirstLearn]

Debuting another new audio release!
Listen to @sarahookr and @cinjoncin talk about Sara's paper - Characterising Bias in Compressed Models.
Find the audio and transcript at https://t.co/gpPbzKwQiL. Some choice quotes in the thread 👇

[Depth First Learning @DepthFirstLearn]

"I grew up in Africa and am interested in making AI accessible in resource constrained environments similar to what I grew up in, but it's also important to understand that we are often making tradeoffs when we talk about properties that we want a model to have. We might not ..."