@DigantaMisra1 @soumikRakshit96 Adaptive Methods for Nonconvex Optimization https://t.co/1MACXhm9vV It's just a simple variant of Adam. You should try it.
@DigantaMisra1 @soumikRakshit96 What's the optimizer module? A new breaking through optimizer like Mish? It's the end of Dec now, cannot wait. My favorite is R(Adam)+Yogi+Lookahead, wish you could compare with it.