Jonathan Jones

Signal processing, computer vision, and machine learning

[Gabriel Peyré @gabrielpeyre]

Oldies but goldies: P. L. Lions and B. Mercier, Splitting Algorithms for the Sum of Two Nonlinear Operators, 1979. Douglas-Rachford algorithm (dual of ADMM) minimizes the sum of two convex functions. A cornerstone of non-smooth optimization. The typical convergence pattern is spiraling. https://t.co/gvHan06uJW

[Chris Paxton @chris_j_paxton]

This point about synthetic data also generally holds true for robotics, in my opinion; lots of areas where we see (1) webscale data on its own is better - capturing semantics - or (2) complex physical interactions, where you need real world data. All 3 are important

[Zeyuan Allen-Zhu, Sc.D. @ZeyuanAllenZhu]

Our 12 scaling laws (for LLM knowledge capacity) are out: https://t.co/qNTarfEb3l. Took me 4mos to submit 50,000 jobs; took Meta 1mo for legal review; FAIR sponsored 4,200,000 GPU hrs. Hope this is a new direction to study scaling laws + help practitioners make informed decisions

[Varun Agrawal @varagrawal]

Excited to share that our paper on a new metric for robot state estimation has been accepted to #ICRA2024.

We (@fdellaert and I) define a unifying metric in the SE_2(3) Lie Group which allows us to combine poses and velocities into a single form.

[William Merrill @lambdaviking]

“The Expressive Power of Transformers with Chain of Thought” w/ @Ashish_S_AI will appear at ICLR 🇦🇹
https://t.co/ay76PikxY0

[Aran Komatsuzaki @arankomatsuzaki]

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Reaches the same performance as Mamba in 2.2x less training steps while preserving the inference performance gains of Mamba against the Transformer

https://t.co/JLMbbO9HsA

[William Merrill @lambdaviking]

🐊📣Still @ NeurIPS?

Come by our poster to hear about how chain of thought/scratchpad steps increase the computational power of transformers

Room 242, 4pm (M3L workshop)

[Dmytro Mishkin 🇺🇦 @ducha_aiki]

Distributed Global Structure-from-Motion with a Deep Front-End

@AyushBaid13, @jlambert_, Travis Driver, @shay_krishnan, Hayk Stepanyan, @fdellaert

tl;dr: GTSfM == parallel extraction & matching, heuristics for cleaning-> new global SfM.
SIFT> deep.
https://t.co/gPvz6VCmk0

[Tri Dao @tri_dao]

Transformers power most advances in LLMs, but its core attention layer can’t scale to long context.

With @_albertgu, we’re releasing Mamba, an SSM architecture that matches/beats Transformers in language modeling, yet with linear scaling and 5x higher inference throughput.
1/

[Najoung Kim 🫠 @najoungkim]

Belated thread on updates to the "Inverse scaling can become U-shaped" paper; so belated that I'm now presenting it at EMNLP :)

Takeaways:
(1) Inv scaling curves are hard to extrapolate
(2) Even minimal demonstrations (1-shot) are critically effective as a mitigation strategy

[Gabriel Peyré @gabrielpeyre]

Oldies but goldies: R. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, 1991. Brockett's flow progressively diagonalizes a symmetric matrix. https://t.co/0XstwPYsT7

[Frank Dellaert @fdellaert]

One interesting find: SIFT is still remarkably good after 25 (30?) years.

[Zhenjun Zhao @zhenjun_zhao]

Robust Self-calibration of Focal Lengths from the Fundamental Matrix

Viktor Kocur, Daniel Kyselica, Zuzana Kúkelová

tl;dr: iterative method->focal length and principal point estimation; degeneracy check for fundamental matrices+RANSAC

https://t.co/Exundg4v6r

[Keenan Crane @keenanisalive]

Suppose I have a collection of identical, non-overlapping, unit-height rectangular blocks in 2D, each given by a real coordinate x and integer coordinate y.

What is the simplest algorithm to check if these blocks are in static equilibrium?

[Sebastian Raschka @rasbt]

"Simplifying Transformer Blocks" ranks easily among my favorite research papers that I've read this year.

Here, the authors look into how the standard transformer block, essential to LLMs, can be simplified without compromising convergence properties and downstream task performance.

Based on signal propagation theory and empirical evidence, they find that many parts can be removed to simplify GPT-like decoder architectures as well as encoder-style BERT models:

skip connections
normalization layers (LayerNorm)
projection and value parameters
sequential attention and MLP sub-blocks (in favor of a parallel layout)

The authors also did a great job referencing tons of related work motivating their experiments. I definitely recommend reading this paper just for the references alone: https://t.co/9nDvsW0yoj

[Simon Schug @ssmonsays]

Why are policy gradient methods typically so sample inefficient?

Whenever a trajectory contains a reward, they credit all actions leading to it. But not all actions mattered! What if we could only credit actions that did matter?

Our NeurIPS spotlight paper does just that 🧵️

[Naomi Saphra @nsaphra]

It's not the first time! A dream team of @enfleisig (human eval expert), Adam Lopez (remembers the Stat MT era), @kchonyc (helped end it), and me (pun in title) are here to teach you the history of scale crises and what lessons we can take from them. 🧵https://t.co/pBUjPNRudf

[Najoung Kim 🫠 @najoungkim]

🦔Announcing SLOG, a new semantic parsing benchmark for structural generalization, to be presented at EMNLP! Work with @BingzhiLi, @lucia_elizabe, @alkoller, @tallinzen, @yuekun_yao.

https://t.co/hmfZezueTo

[Aran Komatsuzaki @arankomatsuzaki]

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-Image Generation

proj: https://t.co/VDt0dSCRer
repo: https://t.co/BR5HEXB6PM
abs: https://t.co/xni5fQQXcH