Jason Taylor

Computer vision researcher

[Jason Taylor @jrbtaylor]

My 5 favorite papers from #ICCV2021 focusing on general principles that fall somewhere between theory and applications:

[Jason Taylor @jrbtaylor]

@fchollet If you're going to troll competing frameworks for fun, at least do a public service and warn people about the horrors of FastAI

[Jason Taylor @jrbtaylor]

"Model soup": after running hyperparameter optimization with a fine-tuned network, you average the weights of several good runs instead of only picking the best one. Similar motivation to stochastic weight averaging, soupier name: https://t.co/IypIVDFiRh

[Jason Taylor @jrbtaylor]

It's unclear whether this improved translation invariance impacts generalization in any other sense as there's no experiments on this. Note that it requires cutting the batch size by half to accommodate the second crops.

[Jason Taylor @jrbtaylor]

My 5 favorite papers from #ICCV2021 focusing on general principles that fall somewhere between theory and applications:

[Jason Taylor @jrbtaylor]

Global Pooling, More than Meets the Eye (https://t.co/lLkjaWFm7e) Position info is not lost with global mean pooling, it's encoded in the channels. For each training image use two different crops and add the MSE of their latent features to the loss

[Jason Taylor @jrbtaylor]

@AlfOKenney Why?

[Jason Taylor @jrbtaylor]

@javaloyML Which term? Hypernetworks are several years old. "Multiplicative interactions" as far as I know are still poorly understood (in terms of capacity, training dynamics, etc.) and there's no work tying together LSTM gates, hypernetworks and attention

[Jason Taylor @jrbtaylor]

@tw_killian @KyleLiang5 @EugeneVinitsky Even worse are the papers that invent something novel, hype it up and attempt to give some theoretical justification when it's just a hack that happened to work on one problem. So there's definitely bad "theory" papers too

[Jason Taylor @jrbtaylor]

@tw_killian @KyleLiang5 @EugeneVinitsky True, but most applied papers aren't written to show what works and what doesn't. 99% don't mention what doesn't work. Too often the focus is "look at this great thing we did" and not why it works. Incremental results with no real takeaway isn't useful for the reader

[Jason Taylor @jrbtaylor]

Cool work combining the ideas of hypernetworks and positional encodings to learn convolutional kernels for sequential tasks

[David W. Romero @davidwromero]

Conv. kernels are often modeled as vectors of independent weights. But, can't we do any better?

In our latest work w/@a_kzna @erikjbekkers @jmtomczak & MH, we see that implicitly representing conv. kernels in CNNs yields several cool advantages!

Code, Demos & Slides available!

[Jason Taylor @jrbtaylor]

I want to see a pair of these as the stereo cameras on a self-driving car. Then the children's movie Cars can finally replace Terminator in discussions about AI #autonomousdriving

[Edge Impulse @EdgeImpulse]

Eyecam is an incredibly realistic robotic eye webcam that can see, blink, and look around like a human: https://t.co/KnIuwI5RDK