Adi Gorla @ ICML 🇰🇷

Currently @MSRNE , ML4Bio PhD Student @ucla | playing with multimodal latent representations and loss functions | prev @genentech

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Gonna be presenting this (poster) at #ICML26 this Thursday@2:30p!
It’ll be at Hall A #2805
Swing by if you’d like learn more limitations of tabular model evals or just wanna chat :)

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Getting scooped on my own paper drop by @DamienTeney 😂

But he's spot on - tabular language models have been overblown

Reported performance gains often don’t come from emergent generalization at all.

https://t.co/kZcXJ5DTAS

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

@tunguz Lol yup, we have a preprint showing just how underdeveloped TLM evals are
https://t.co/p3s2Qbhx81

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

@predict_addict Thanks for sharing our work :)

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

@ducha_aiki @ratishsp @tunguz Thanks for this! The skepticism (at least for TLMs) was was spot on

[Dmytro Mishkin @ducha_aiki]

The Illusion of Generalization: Re-examining Tabular Language Model Evaluation

Aditya Gorla, @ratishsp

tl;dr: @tunguz and other Kaggle GMs were right all along about tabular DL models.
https://t.co/fzOECv0cWA

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

@DamienTeney Hahah no apologies needed! took it as a big compliment

Really appreciate you sharing it and the kind words :)

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Getting scooped on my own paper drop by @DamienTeney 😂

But he's spot on - tabular language models have been overblown

Reported performance gains often don’t come from emergent generalization at all.

https://t.co/kZcXJ5DTAS

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Most "learning" is either predicting majority class, memorizing contaminated data, or generic instruction-following. Not tabular-specific at all.

Benchmarks need baselines, better dedup, task stratification, and cleaner eval datasets.

Really fun working with @ratishsp on this!

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Some major findings:

1) 39% of datasets show zero improvement over majority class baseline

2) Top performers severely contaminated; complete train-test overlap that evades deduplication

3) Much of the gain comes from instruction tuning alone

So whats actually being learned?

[Na Cai @caina89]

Our perspective on heritable confounders in big GWAS meta-analyses on psychiatric disorders is now online @NatureGenet: https://t.co/Nkh1ejGcun, with @andywdahl, @jonathan_flint1, Richard Border et al. Small thread for context 1/n

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

@JingyouR @hjpimentel Congrats!!

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

This work would not have been possible without my amazing collaborators Ryan Wang @ZhengtongLiu @ulzeean and advisor @sr_sankararaman!
9/9

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Some exciting news: Our method CACTI will be a ✨spotlight ✨poster at #ICML2025!
CACTI solves the missing data problem in tabular datasets with a transformer that learns from REAL missingness patterns + semantic context, beating SOTA by 13.4% in the most realistic settings
🧵1/9

[Adi Gorla @ ICML 🇰🇷 @ag0rla]

Come check out poster at #icml2025 in East Exhibition Hall A-B: E-1300 at the 11am session on 16/07 : https://t.co/lb86sKAR8x
Or read our preprint for more details: https://t.co/BTeTLC2ZzF
8/9