Yura Gorishniy

I do research on deep learning for tabular data

[Yura Gorishniy @YuraFiveTwo]

TabM now has a Python package!

TabM is a simple and powerful DL architecture for tabular data that efficiently imitates an ensemble of MLPs

🏆 TabM has been used in winning solutions on Kaggle, and performs well on TabReD -- a challenging benchmark!

💻 pip install tabm

👇Link

[Yura Gorishniy @YuraFiveTwo]

Detail 2: we don't even mention this in the text, but weight decay tuning spaces in this technical report are improved compared to our prior work (the spaces vary between models).

That's it for today! No big plans for this technical report, but hopefully it will come handy :)

[Yura Gorishniy @YuraFiveTwo]

A small yet practical update for tabular deep learning people:

Muon is a strong alternative to AdamW for training modern tabular MLPs, including TabM. Give it a try!

Overall, our technical report covers 15 optimizers:
https://t.co/e4uBFuqVhB

Details 👇

[Yura Gorishniy @YuraFiveTwo]

Detail 1: besides Muon, we also found AdamW+EMA to perform well for _plain_ MLPs. For stronger MLP-based models, such as TabM, Muon performs better.

[Dmitry Eremeev @eremeev_d42]

Graph foundation model with SOTA results on real-world graphs!

Our “GraphPFN: A Prior-Data Fitted Graph Foundation Model” paper recently got a major update, with better ICL performance, new ablations, code improvements and more!

🧵1/11

[Kirill Mazur @makezur]

Introducing 4D Primitive-Mâché (4DPM), a new method for replayable 4D reconstruction from monocular videos.

We split dynamic scenes into 3D primitives and recover their motion. 4DPM can infer object positions even after they leave view.

Joint work with @marwan_ptr @AjdDavison

[Yura Gorishniy @YuraFiveTwo]

@vnjogani @akshay_pachaar Hi, I am one of TabM authors. In the paper, we don't do any kind of subsampling during training, though this is definitely possible. As for the "adapters", they perform elementwise operations, i.e. they are linear transformations, but not in the sense of torch.nn.Linear.

[Yura Gorishniy @YuraFiveTwo]

@kanpuriyanawab @_avichawla Hi! TabM has been used in winning solutions in recent Kaggle competitions, for example:

(1) https://t.co/wJLrFSmSoJ

(2) https://t.co/mQa7xaNGL5

Yesterday, I shared a Python package for TabM to make it easier to try in practice:

https://t.co/Ys63bMOJ6K

[Yura Gorishniy @YuraFiveTwo]

TabM now has a Python package!

TabM is a simple and powerful DL architecture for tabular data that efficiently imitates an ensemble of MLPs

🏆 TabM has been used in winning solutions on Kaggle, and performs well on TabReD -- a challenging benchmark!

💻 pip install tabm

👇Link

[Yura Gorishniy @YuraFiveTwo]

@heptoop @_avichawla Hi, I am one of the TabM authors.

The size of the shared MLP is actually "standard", but it is reused across k MLPs (k=32 in the paper). Plus each of the k MLPs has a little amount of non-shared weights. Also, see the new illustration in the linked tweet

https://t.co/Ys63bMOJ6K

[Yura Gorishniy @YuraFiveTwo]

TabM now has a Python package!

TabM is a simple and powerful DL architecture for tabular data that efficiently imitates an ensemble of MLPs

🏆 TabM has been used in winning solutions on Kaggle, and performs well on TabReD -- a challenging benchmark!

💻 pip install tabm

👇Link

[Yura Gorishniy @YuraFiveTwo]

@AlexShtf I was glad to chat, thanks for coming by!

[Yura Gorishniy @YuraFiveTwo]

@hydantess1993 @JFPuget That's great to hear, thank you for sharing!

[Yura Gorishniy @YuraFiveTwo]

TabM now has a Python package!

TabM is a simple and powerful DL architecture for tabular data that efficiently imitates an ensemble of MLPs

🏆 TabM has been used in winning solutions on Kaggle, and performs well on TabReD -- a challenging benchmark!

💻 pip install tabm

👇Link

[Yura Gorishniy @YuraFiveTwo]

@JFPuget Hi! Here are the links:

(1) https://t.co/wJLrFSmSoJ

(2) https://t.co/mQa7xaNGL5

They can also be found at the very beginning of README along with other practical notes

[Yura Gorishniy @YuraFiveTwo]

@fabianjkrueger @_avichawla Hi! TabM now can be installed via pip.

The package still requires familiarity with PyTorch, but hopefully the Colab example will make it easier to get started

https://t.co/Ys63bMOJ6K

[Yura Gorishniy @YuraFiveTwo]

TabM now has a Python package!

TabM is a simple and powerful DL architecture for tabular data that efficiently imitates an ensemble of MLPs

🏆 TabM has been used in winning solutions on Kaggle, and performs well on TabReD -- a challenging benchmark!

💻 pip install tabm

👇Link

[Yura Gorishniy @YuraFiveTwo]

Note: the package requires familiarity with PyTorch

To help users get started, we provide a Jupyter notebook with an end-to-end example of training TabM:

https://t.co/JDCRCXEwWE

[Yura Gorishniy @YuraFiveTwo]

https://t.co/dhhP5nuQjg

The package makes efficient ensembles for tabular data more accessible by providing:

🤖 TabM
🔧 Layers for building custom TabM-like models
✨ Functions for turning existing models into efficient ensembles

The screenshot covers all three use cases :

[Yura Gorishniy @YuraFiveTwo]

@felixo_dmv @_avichawla Hey, I am one of the TabM authors. TabM is basically an efficiently implemented ensemble of MLPs, i.e. a somewhat generic architecture. So it should be applicable in many different contexts, just like MLP itself. Though we did not benchmark TabM on time series.

[Yura Gorishniy @YuraFiveTwo]

@wappledoobie @k_adeyemiai @_avichawla Hey, I am one of the authors. In the paper we report inference throughput on CPU and GPU. Since TabM is just a bunch of MLPs, it is surely slower than one plain MLP, but still practical and hardware-friendly. Furthermore, the number of MLPs can be greatly reduced, see Section 5.2

[Dmitry Baranchuk @DmitryBaranchuk]

I'd like to share our new diffusion distillation method, SwD, which produces few-step generators with progressive resolution scaling over the diffusion process. On SD3.5, SwD matches the speed of two full-size steps but with much better quality. Demo & models are released. (1/9)

[Anastasiia Koloskova @Ana_koloskova]

I am excited to announce that I will join the University of Zurich as an assistant professor in August this year! I am looking for PhD students and postdocs starting from the fall.

My research is on optimization, federated learning, machine learning, privacy, and unlearning.