foobarbare

[Eugénie Bastié @EugenieBastie]

Est-il normal qu’une génération entière vive en moyenne, mieux que les autres ? Rappelons quelques faits. C’est la première fois de l’histoire de France que ceux qui ne travaillent plus sont plus riches que ceux qui travaillent. Les retraités ont un niveau de vie supérieur à l’ensemble de la moyenne de la population. C'est unique dans l'OCDE. Les retraités actuels, ceux de la génération baby-boom touchent en moyenne deux fois plus que ce qu’ils ont cotisé. Les millenials toucheront probablement moins que ce qu’ils ont cotisé dans leur vie. Il y a là une rupture d’équité.

Ceux qui promettent le fantasme d’un retour à la retraite à 62 ans sont des menteurs ou des insensés. C’est tout notre système par répartition qui est profondément injuste. Tout cela était prévisible, mais personne n’y touchera jusqu’à ce que nous soyons au pied du mur.

Parce que les retraités sont ceux qui en France font les élections. Dans un pays vieillissant, le système politique est naturellement tourné vers les seniors. Nous glissons doucement vers une forme de gérontocratie.

[Demis Hassabis @demishassabis]

Thrilled to announce AlphaFold 3 which can predict the structures and interactions of nearly all of life’s molecules with state-of-the-art accuracy including proteins, DNA and RNA. Biology is a complex dynamical system so modeling interactions is crucial https://t.co/Gs4GoOB3fD

[Dmytro Mishkin 🇺🇦 @ducha_aiki]

FeatUp: A Model-Agnostic Framework for Features at Any Resolution

@xkungfu, Mark Hamilton, Laura Brandt, Axel Feldman, Zhoutong Zhang, William T. Freeman

tl;dr: in title. Upscale DINOv2, anyone?
https://t.co/AVpWfn9Vpy
#ICLR2024

[Andrej Karpathy @karpathy]

Beautiful work / attention to detail trying to get Gemma to finetune correctly. There are so many foot guns here to be super careful with. All of these issues don't throw any errors, they silently make your network worse.

A great example of what I wrote about in my "A Recipe for Training Neural Networks":
"""The "fast and furious" approach to training neural networks does not work and only leads to suffering. Now, suffering is a perfectly natural part of getting a neural network to work well, but it can be mitigated by being thorough, defensive, paranoid, and obsessed with visualizations of basically every possible thing. The qualities that in my experience correlate most strongly to success in deep learning are patience and attention to detail."""

And why I so emphasize the need for understanding all the parts of the deep learning stack in great detail. I exist in a perpetually terrified state of the remaining 20 silent bugs that certainly remain in my code.

[Jerome Revaud @JeromeRevaud]

An example of how DUSt3R can do "impossible matching": given two images without any shared visual content (my office, obviously never seen at training), it can output an accurate reconstruction (no intrinsics, no poses!) in seconds

[Gwangbin Bae @BaeGwangbin]

Excited to introduce 𝗗𝗦𝗜𝗡𝗘! (#CVPR2024)

https://t.co/DaWRX8e8Mw

We push the limits of single-image surface normal estimation by rethinking the inductive biases needed for the task.

See you in Seattle!

[Niels Rogge @NielsRogge]

SegGPT is now available in @huggingface Transformers! An elegant approach for image segmentation given a single (image, mask) prompt, GPT-like. 🙌

Docs: https://t.co/XuPCEUmmKk

Notebook: https://t.co/nPDJxwu4cJ

[Shubham Tulsiani @shubhtuls]

[1/6] What representation comes to mind when you think of a ‘camera’? Perhaps an extrinsic + intrinsic matrix? In our ICLR (oral) paper, we instead infer a distributed representation where each pixel is associated with a ray, and show SoTA results for few-view pose estimation.

[Matt Shumer @mattshumer_]

Damn, Gemma is really stubborn.

[OpenAI @OpenAI]

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.

https://t.co/YYpOAcrXQ3

Prompt: “Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous sakura petals are flying through the wind along with snowflakes.”

[AK @_akhaliq]

TikTok presents Depth Anything

Unleashing the Power of Large-Scale Unlabeled Data

paper page: https://t.co/fgga7kjUfb
demo: https://t.co/I7x17X4Qsi

Depth Anything is trained on 1.5M labeled images and 62M+ unlabeled images jointly, providing the most capable Monocular Depth Estimation (MDE) foundation models with the following features:

zero-shot relative depth estimation, better than MiDaS v3.1 (BEiTL-512)

zero-shot metric depth estimation, better than ZoeDepth

optimal in-domain fine-tuning and evaluation on NYUv2 and KITTI

[AK @_akhaliq]

Microsoft presents Contrastive Preference Optimization

Pushing the Boundaries of LLM Performance in Machine Translation

paper page: https://t.co/oP5x4XeD2R

Moderate-sized large language models (LLMs) -- those with 7B or 13B parameters -- exhibit promising machine translation (MT) performance. However, even the top-performing 13B LLM-based translation models, like ALMA, does not match the performance of state-of-the-art conventional encoder-decoder translation models or larger-scale LLMs such as GPT-4. In this study, we bridge this performance gap. We first assess the shortcomings of supervised fine-tuning for LLMs in the MT task, emphasizing the quality issues present in the reference data, despite being human-generated. Then, in contrast to SFT which mimics reference translations, we introduce Contrastive Preference Optimization (CPO), a novel approach that trains models to avoid generating adequate but not perfect translations. Applying CPO to ALMA models with only 22K parallel sentences and 12M parameters yields significant improvements. The resulting model, called ALMA-R, can match or exceed the performance of the WMT competition winners and GPT-4 on WMT'21, WMT'22 and WMT'23 test datasets.

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Generative large language models (LLMs) are based upon the decoder-only transformer architecture. Currently, these types of generative LLMs are incredibly popular. However, I use encoder-only architectures for 90% of use cases as a practitioner. Here’s why…

History of encoder-only models. The encoder-only transformer architecture was popularized by the proposal of BERT in 2018. At the time of its proposal, BERT set a new state-of-the-art performance on every natural language task that was considered in its publication. For this reason, BERT revolutionized research in natural language processing, replacing many domain-specific techniques with a single model that can solve nearly all tasks!

Encoder-only architecture. Although the original transformer architecture contains both an encoder and a decoder, BERT leverages an encoder-only architecture. The encoder-only architecture just contains several repeated layers of bidirectional self-attention and a feed-forward transformation, both followed by a residual connection and layer normalization. The original encoder-only BERT models that were proposed have the following sizes:

- BERT Base: 12 layers, 768-dimensional hidden representations, 12 attention heads in each self-attention module, and 110M parameters.
- BERT Large: 24 layers, 1024-dimensional hidden representations, 16 attention heads in each self-attention module, and 340M parameters.

Notably, BERT Base is the same size as the original GPT model. In other words, these models are significantly smaller (and therefore easier to manage/deploy!) compared to the generative LLMs that are popular today.

BERT pretraining. Similar to generative LLMs, BERT has an extensive pretraining process. Instead of next token prediction, however, we pretrain BERT using a Cloze objective, which randomly masks out words/tokens from the input and tries to predict them. Because BERT uses bidirectional self-attention (instead of masked self-attention, which is used by decoder-only models), the model can look at the entire sequence both before and after the masked token to make a prediction.

Using BERT in practice. To use BERT to solve a practical task, we simply finetune the model over task-specific data. In particular, BERT is very good at solving sentence and token-level classification tasks. Additionally, extensions of BERT (e.g., sBERT) can be used for semantic search, making BERT applicable to retrieval tasks as well. In general, finetuning BERT is easy/efficient and yields high performance even with small amounts of training data.

What can’t we do? Encoder-only (BERT) models are small, use bidirectional self-attention, and can be easily fine-tuned to impressive performance. As such, finetuning BERT to solve classification tasks is oftentimes preferable to performing few-shot prompting via an LLM, assuming we have the ability to train models and a little bit of training data. However, encoder-only models cannot generate text, so we can only use them for solving discriminative tasks.

[Zhuang Liu @liuzhuang1234]

How to choose a vision model for your specific needs?

How do ConvNet / ViT, supervised / CLIP models compare with each other on metrics beyond ImageNet?

Our work comprehensively compares common vision models on "non-standard" metrics. (1/n)

[National Institute of Standards and Technology @NIST]

Adversaries can deliberately manipulate AI systems to make them malfunction — and there’s no foolproof defense.

Check out NIST’s new guide to the types of attacks developers and users can expect, along with approaches to mitigate them: https://t.co/AhbluzSeEx

[Andrew Davison @AjdDavison]

Worth a read: What kind of bubble is AI? By Cory Doctorow, via @rodneyabrooks. i.e. how much value will be left when the current LLM-driven hype subsides.
https://t.co/46hs9CPd8F

[Gabriel Peyré @gabrielpeyre]

Optimal Transport (OT) imposes the exact conservation of mass, which is often problematic. Unbalanced OT relaxes this constraint and makes OT robust to outliers and noise. 1/7

[Hyung Won Chung @hwchung27]

New paper + models!

We extend instruction finetuning by
1. scaling to 540B model
2. scaling to 1.8K finetuning tasks
3. finetuning on chain-of-thought (CoT) data

With these, our Flan-PaLM model achieves a new SoTA of 75.2% on MMLU.

[foobarbare @foobarbare]

Image matching on really difficult image pairs, is not at HLP.
And these are buildings... try reflective objects, with repeated patterns and little "keypoints"

[Dmytro Mishkin 🇺🇦 @ducha_aiki]

11/10
Here DISK results. DISK is quite cool
@jatentaki

[foobarbare @foobarbare]

DEGENSAC in OpenCV https://t.co/OKdQx69kLu