BRAIn Lab ร MBZUAI โ 5 years of collaboration ๐ฅ
Together, weโve prepared a survey on Byzantine attacks โ link below ๐
https://t.co/sNGzIN0K5L
Despite a challenging period in Abu Dhabi, the work continues. Supporting our colleagues ๐
To get more details๐๐ป
๐ Paper: https://t.co/mOhT8fwWEi
๐๏ธ References:
[1] https://t.co/rxHD8xjE0e
[2] https://t.co/WoVDUw4qLi
[3] https://t.co/Anw3bZp5ox
Adaptive step sizes without knowing the smoothness constant? Thatโs real! Compatible with ANY variance-reduction method โ finite-sum, distributed, coordinate sketches included
๐ Unified Theory of Adaptive Variance Reduction
From pure SGD to advances like variance reduction
Struggling with data heterogeneity in your deep learning projects? We introduce ALSO, a new, practical Distributionally Robust Optimizer (DRO)๐
๐ Paper thread for "Aligning Distributionally Robust Optimization with Practical Deep Learning Needs"
We tested ALSO across diverse setups with real-world data heterogeneity, including:
โข Class-imbalanced datasets
โข Tabular DL
โข Robust & distributed training
โข Split learning
In all scenarios, ALSO showed superior performance over both standard approaches and DRO baselines