Looking forward to presenting our poster at ICML this month!
"Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens" with @jmhernandez233 – what could we learn by applying heuristics from second-order optimisers to first-order methods?
https://t.co/htLmtSvpXO
2: "Adam through a Second-Order Lens" with Baiyu Su and @jmhernandez233 – applying K-FAC's damping and learning rate selection logic to Adam gives AdamQLR: an efficient hybrid optimiser whose default hyperparameters compete with tuned SGD/Adam
https://t.co/htLmtSvpXO
Delighted to share two new papers on arXiv!
1: "Series of Hessian-Vector Products for Tractable Saddle-Free Newton Optimisation of Neural Networks" with @cravingoxygen@jmhernandez233 – saddle-point avoiding, asymptotically exact second-order NN training
https://t.co/RrlDTCJWB3
We're excited to be at #ICLR2022 this week! Our group and collaborators will be presenting 11 papers (including an Oral and 3 Spotlights) at the main conference!
Check out our @iclr_conf schedule below. Please drop by and have a chat.