Alexander Chebykin

PhD student researching Deep Learning + Evolutionary Algorithms @ CWI

[Alexander Chebykin @a__chebykin]

🎉 Our paper "To Be Greedy, or Not to Be — That Is the Question for Population Based Training Variants" is accepted to TMLR!

TL;DR Bayesian PBTs optimize the greedy objective more effectively than non-Bayesian PBTs, this can be good or bad (depends on the task & hyperparams)
1/7

[Alexander Chebykin @a__chebykin]

P.S. Thank you to my supervisors and coauthors, @TanjaAlderlies1 and @PeterANBosman

[Alexander Chebykin @a__chebykin]

🎉 Our paper "To Be Greedy, or Not to Be — That Is the Question for Population Based Training Variants" is accepted to TMLR!

TL;DR Bayesian PBTs optimize the greedy objective more effectively than non-Bayesian PBTs, this can be good or bad (depends on the task & hyperparams)
1/7

[Alexander Chebykin @a__chebykin]

Check out the paper for details!
https://t.co/YCKN9XTZor

We also release our code containing task-agnostic implementations of five PBT variants, hopefully making future research and comparison of PBT variants easier! 7/7
https://t.co/vKFqscU3Ef

[Alexander Chebykin @a__chebykin]

@MaciejSzankin @aiexplorations @rasbt In my opinion, the raw number of combinations doesn't matter if many of them give ~the same results. If we care about discovering breakthrough architectures, just recombining conv flavours/pruning is the wrong way to go, no matter how "big" the space is

[Alexander Chebykin @a__chebykin]

@aiexplorations @rasbt Many papers do NAS with evolutionary algorithms, but the improvements are small because the computationally feasible search spaces are too simple and restrictive - and that limits any algorithm

[Alexander Chebykin @a__chebykin]

@joelbot3000 I believe the most recent essay of @rootsofprogress addresses this (https://t.co/c7yvTIiJYT): "But this form of progress is not an end in itself. True progress is advancement toward the good, toward ultimate values—call this “ultimate progress,” or “progress in outcomes.”"

[Alexander Chebykin @a__chebykin]

@RobertTLange @GoogleDeepMind @alanyttian @yujin_tang Interesting work, though SNES apparently is still undefeated in the Neuroevolution tasks. I'm wondering if combining LLMs with GAs would make more sense (the parametric nature of ES could be restricting the usefulness of LLMs)

[Arkadiy Dushatskiy 🇺🇦 @ADushatskiy]

Excited to share our #ICML2023 paper, in which we introduce a state-of-the-art parallel scalable algorithm for Multi-Objective hyperparameter optimization! We build upon Population Based Training (PBT), so simply call it ��𝐎-𝐏𝐁𝐓. 🧵 1/N

[Arkadiy Dushatskiy 🇺🇦 @ADushatskiy]

Super excited about our paper "Multi-Objective Population Based Training" (with @a__chebykin, @TanjaAlderlies1 and @PeterANBosman) accepted for the @icmlconf! The title is fairly self-explanatory but of course arxiv link and🧵 coming soon! #ICML2023

[Alexander Chebykin @a__chebykin]

@barneyp @mlpowered @andrey_kurenkov @AnthropicAI I was reminded of this (https://t.co/lb6iZWdPT9) very interesting write-up on grokking, where the author showed that despite the total loss decreasing abruptly, the cause is the gradual decrease of loss of different circuits

[Alexander Chebykin @a__chebykin]

P.S. Huge thanks to my supervisors @PeterANBosman and @TanjaAlderlies1

[Alexander Chebykin @a__chebykin]

Our paper “Evolutionary Neural Cascade Search across Supernetworks” received the Best Paper Award in the Neuroevolution track @ GECCO 2022!🥳

TL;DR
Cascades + NAS + many supernets = SOTA NAS for efficient models
Cascades + 518 timm models = Dominating ImageNet Pareto front

🧵👇

[Alexander Chebykin @a__chebykin]

Conclusion: cascading could improve your trade-off front. Give it a try!
Code: https://t.co/ICjZvSpaGi
Blog: https://t.co/6V4vzMZTOT
Paper: https://t.co/a3ZUeMagWl