YK Community Feed

Community-sourced ML research feed.
Submissions:

[YK Community Feed @yk_feed]

A simple conceptual framework for improving, understanding and deriving self-supervised learning methods!
https://t.co/oGexFJHdIa

[Yann Dubois @yanndubs]

#NeurIPS2022
What are ideal representations for self-sup. learning (SSL)?

🤓We give simple optimality conditions and use them to improve/understand/derive SSL methods!

🔥outperform baselines on ImageNet

https://t.co/SbFaePX2Z0
w. @tatsu_hashimoto @StefanoErmon @percyliang
🧵

[YK Community Feed @yk_feed]

A simple conceptual framework for improving, understanding and deriving self-supervised learning methods!
https://t.co/oGexFJYP6K

[Yann Dubois @yanndubs]

#NeurIPS2022
What are ideal representations for self-sup. learning (SSL)?

🤓We give simple optimality conditions and use them to improve/understand/derive SSL methods!

🔥outperform baselines on ImageNet

https://t.co/SbFaePX2Z0
w. @tatsu_hashimoto @StefanoErmon @percyliang
🧵

[YK Community Feed @yk_feed]

A new approach to tabular learning: Bayesian approximation + Transformers + Causality = SOTA performance in < 1 second
https://t.co/iWasnPHipe

[Frank Hutter @FrankRHutter]

This may revolutionize data science: we introduce TabPFN, a new tabular data classification method that takes 1 second & yields SOTA performance (better than hyperparameter-optimized gradient boosting in 1h). Current limits: up to 1k data points, 100 features, 10 classes. 🧵1/6

[YK Community Feed @yk_feed]

New equilibria can be defined that are not subsets of Nash but instead generalize superrationality. They are at most unique and Pareto-optimal. A first analysis shows that social utility metrics are higher than those of Nash for the studied games.

[YK Community Feed @yk_feed]

As Machine Learning predictions become more accurate, the assumptions behind Nash game theory are pushed to the limits of their validity because of the prediction<->decision feedback loop. Game theory needs to be adapted with a shift of paradigm.

[YK Community Feed @yk_feed]

In addition, Common Knowledge of Rationality (I know they know I know... they are rational) is extended to the more stringent assumption of Necessary Rationality: we are all rational in all possible worlds, even if our decisions had been different.

[YK Community Feed @yk_feed]

Hierarchical VAEs continue to improve.
https://t.co/1VeDtOhLhK

[Rayhane Mama @RayhaneMama]

Happy to finally share our work!

Joint collaboration with Louay Hazami and Ragavan Thurairatnam.

Efficient-VDVAE obtain SOTA log-likelihood on 7 commonly used image datasets. We make an argument against 5-bit benchmarks and lay some ground work for downstream tasks with HVAEs.

[YK Community Feed @yk_feed]

Controllable symbolic music generation!
https://t.co/rgkIz4vcLF

[AK @_akhaliq]

FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control
abs: https://t.co/cv551j9KqJ

[YK Community Feed @yk_feed]

Multi-modality continues to advance with CM3!
https://t.co/LBaO1tzE1J

[Armen Aghajanyan @ArmenAgha]

I’m excited to present our paper CM3: A Causal Masked Multimodal Model of the Internet where we train a model that can do zero-shot unconditional/conditional image generation (PixelCNN/DALL-E), image-infilling/captioning, entity linking/disambig, summarization all with prompting!

[YK Community Feed @yk_feed]

.@robintibor et al. propose a new method that "simplifies" images, allowing better dataset condensation and debugging and failures!
https://t.co/D0oborbBN0

[Robin Tibor Schirrmeister @robintibor]

Are all bits useful? How can we simplify images, and how do “simpler” images affect neural network training?🤔 with @sarahookr @savvyRL Tonio Ball
Paper: https://t.co/vUiljFOQ4j
Code: https://t.co/JjcsWvf4sX
🧵

[YK Community Feed @yk_feed]

.@BlancheMinerva evaluates how well GPT-J plagiarizes by investigating MOSS' "plagiarism score" and finds that it outperforms humans.
https://t.co/ixLw6fYpYI

[Stella Biderman @BlancheMinerva]

Excited to share my newest paper, "Neural Language Models are Effective Plagiarists" with @EdwardRaffML. We took a dataset of CS 101 assignments and asked "can a language model do a good job solving these with minimal human intervention or knowledge?"

https://t.co/LcinStQ4ag

[YK Community Feed @yk_feed]

Put your documents in a new light :)
https://t.co/2zoxgLXOWA

[Austin Huang @austinvhuang]

We had the privilege of presenting our Sim2Real Docs project at @NeurIPSConf 2021 DCAI.

TLDR: Python library for synthetic images of documents in natural scenes using @Blender.

It's open source:
https://t.co/47UQecCmfw

It's @nmaddikunta21's first paper:
https://t.co/tDI1eQ6WfG

[YK Community Feed @yk_feed]

Put your documents in a new light :)
https://t.co/2zoxgLGe52

[Austin Huang @austinvhuang]

We had the privilege of presenting our Sim2Real Docs project at @NeurIPSConf 2021 DCAI.

TLDR: Python library for synthetic images of documents in natural scenes using @Blender.

It's open source:
https://t.co/47UQecCmfw

It's @nmaddikunta21's first paper:
https://t.co/tDI1eQ6WfG

[YK Community Feed @yk_feed]

distribution of value at every time step, instead of attempting to solve everything by taking only the expectation, which gives a very narrow view of the environment. Related works to this are QR DQN and TQC.

[YK Community Feed @yk_feed]

When I discovered this paper and it's related papers, I thought it would have much higher reach than what I'm currently seeing, seeing that it's a more elegant way of thinking about the Bellman Equation. At it's core, DRL directly models the

[YK Community Feed @yk_feed]

Grokking is now on ArXiv!
https://t.co/x9QXn8Lmfc

[AK @_akhaliq]

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
abs: https://t.co/AjYuSpyzU3

NNs learn through a process of "grokking" a pattern in the data, improving generalization performance from random chance level to perfect generalization

[YK Community Feed @yk_feed]

.@jonLorraine9 et al conduct an in-depth analysis of optimizers and propose a new one that incorporates a "branching" mechanic.
https://t.co/7LZu0fXwgV

[Jonathan Lorraine @jonLorraine9]

When minimizing objectives, randomly initializing + optimizing can fail to find different solutions. We give a branching optimization method for this using Lyapunov exponents..1/11
https://t.co/WjL7s3DT8X @AAMAS2022
With @PaulVicol @jparkerholder @TalKachman @Luke_Metz @j_foerst

[YK Community Feed @yk_feed]

@jonLorraine9 et al conduct an in-depth analysis of optimizers and propose a new one that incorporates a "branching" mechanic.
https://t.co/7LZu0fXwgV

[Jonathan Lorraine @jonLorraine9]

When minimizing objectives, randomly initializing + optimizing can fail to find different solutions. We give a branching optimization method for this using Lyapunov exponents..1/11
https://t.co/WjL7s3DT8X @AAMAS2022
With @PaulVicol @jparkerholder @TalKachman @Luke_Metz @j_foerst