Theo Rekatsinas

Machine Learning Systems, Data Management & Knowledge Graphs @Apple; Ex-Professor @ETH & @UWMadison; Co-founder of inductiv (acquired by @Apple)

[Theo Rekatsinas @thodrek]

Scalability is a key factor limiting the use of Graph Neural Networks (GNNs) over large graphs; w/ @RWaleffe, @JasonMohoney , and Shiv, we introduce Marius++ (https://t.co/w4Hots8T7d), a system for *out-of-core* GNN mini-batch training over billion-scale graphs. (1/5)

[Theo Rekatsinas @thodrek]

@DimitrisPapail https://t.co/kQP1HGf4E8

[Elon Musk @elonmusk]

Please stop using hashtags. The system doesn’t need them anymore and they look ugly.

[Theo Rekatsinas @thodrek]

@andrew_ilyas @aleks_madry @KonstDaskalakis Big congratulations Andrew! Exciting next steps!

[Theo Rekatsinas @thodrek]

@DimitrisPapail Nobel prize

[Logan Engstrom @logan_engstrom]

Announcing a deadline extension for the ATTRIB workshop! Submissions are now due September 25th, with an option to submit October 4th if at least one paper author volunteers to be an emergency reviewer. More info here: https://t.co/a2TEiG1FLP

[Theo Rekatsinas @thodrek]

@JanaGiceva @ERC_Research @tum_db @TU_Muenchen Big congratulations!

[Theo Rekatsinas @thodrek]

@DimitrisPapail @MSFTResearch Congratulations!

[Pınar Tözün (@[email protected]) @pinartozun]

We had a great @dagstuhl seminar this week gathering to discuss resource-efficient ML. Many thanks to my co-organizers (@ppietzuch, @matthiasboehm7, @anaklimovic, @oanabalmau) and all the participants!

[Theo Rekatsinas @thodrek]

@madsjw Big congratulations!! Well deserved!

[Theo Rekatsinas @thodrek]

Time for #AI and real-time analytics to take the fun out of cycling https://t.co/lLpvr2oLS7

[Theo Rekatsinas @thodrek]

Very cool work @RWaleffe !!

[Bryan Catanzaro @ctnzr]

A 8B-3.5T hybrid SSM model gets better accuracy than an 8B-3.5T transformer trained on the same dataset:
* 7% attention, the rest is Mamba2
* MMLU jumps from 50 to 53.6%
* Training efficiency is the same
* Inference cost is much less
https://t.co/x62otbC5uN

[Theo Rekatsinas @thodrek]

@DataCereal @cloudera @VertaAI Congratulations Manasi!

[Peter Lee @peteratmsr]

Aurora is an AI foundation model that flexibly achieves SoTA 5-day air pollution, 10-day global weather, and other forecasts. More importantly, it shows evidence of good scaling properties and adaptation to new atmospheric tasks. Excited to see scale up. https://t.co/Swgbn0q6Sp

[Theo Rekatsinas @thodrek]

@ImmanuelTrummer @Cornell @CornellCIS Congratulations!!! 🚀

[Theo Rekatsinas @thodrek]

☘️💚 once again to the top!!! @kos_slou #euroleague @paobcgr

[Theo Rekatsinas @thodrek]

@HongyiWang10 @RutgersCS Congratulations Hongyi!

[Theo Rekatsinas @thodrek]

@s_batzoglou @GaryMarcus maybe we are not feeling the AGI

[Theo Rekatsinas @thodrek]

@BlackHC @RWaleffe @vmageirakos The methods are related but one key point in our work is that of adjusting the learning rate schedule. Also, our focus was on studying the true compute costs of prior "practical" methods. Yet, in theory selection can be better than full-dataset training: https://t.co/HAQpZrjHXF

[Theo Rekatsinas @thodrek]

Data pruning to reduce pertaining costs is hot, but fancy pruning can take just as long to select data as to train on all of it! Patrik, @Rwaleffe, and @vmageirakos's work at #ICLR2024 tomorrow shows how a simple, low-cost tweak to random sampling outperforms trendy methods!

[Roger Waleffe @RWaleffe]

Not convinced about using random sampling for data pruning? Consider twice! In our recent work, we introduce Repeated Sampling of Random Subsets: https://t.co/jk2dWHpocl, where we sample a subset of data at each epoch of training instead of only once at the beginning!

[Theo Rekatsinas @thodrek]

You can find all our comparisons against 30+ importance-based data pruning and selections methods at our paper: https://t.co/a9JnrBkKuI Turns out that sophisticated pruning might be a mirage for pre-training...

[Besmira Nushi 💙💛 @besanushi]

New job post looking for senior ML Engineers in Model Evaluation and Understanding. If you are at #ICLR2024, come talk to our group at the Microsoft Booth tomorrow at 9:30 am. Link for application: https://t.co/iYw4BBxAhz