@RAPIDSai To sum it up:
1. Evaluation = speed
2. GPUs everywhere
3. Limit your architectures
4. Kaggle solutions are a DB
5. Refactor more
6. Pipelines > models
I hope you learned something!
Follow me @marktenenholtz to get more high-signal ML content!
I've spent 1000's of hours building ML models over the last couple of years.
Here are some tips that would have made me work 10x faster (that you can read in 2 minutes):
TL;DR:
Tabular: XGBoost/LightGBM/RF
Time series: XGBoost/LightGBM/RF
Image: ResNet/EffNet
Text: RoBERTa
Audio: ResNet/EffNet
Your best bet is usually to start with these and then experiment from there.
Nothing in ML is an end-all-be-all!