Qiang Liu

Associate professor at UT Austin, machine learning.

[Qiang Liu @lqiang67]

🚀 New Rectified Flow materials (WIP)!

📖 Tutorials: https://t.co/8RBpAxW9an
💻 Code: https://t.co/qHkFsihIFA
📜 Notes: https://t.co/5wIf58CACc

Contributions from @RunlongLiao, @XixiHu12, @cranialxix, and many others! 🔥

Let us know your thoughts! 🚀

[Qiang Liu @lqiang67]

In preference alignment, we train a good model with both positive D+ and negative D− data.

But MLE on D+ alone is the most statistically efficient way to model D+. So why is D- relevant and useful?

Puzzling over this led us to new methods with significant gains—check it out!

[Junbo Li @ljb121002]

Excited to introduce Prior-Informed Preference Alignment (PIPA)🎶!

🚀Works anywhere DPO/KTO does, with a 3-10% performance boost in GSM8K/MATH.

👏Seamlessly unifies 4 data settings with 1 framework: paired/unpaired data × answer/step-wise labels.

💨No extra training cost.

[Kaizhao Liang @KyleLiang5]

TLDR: 1⃣ line modification, satisfaction (theoretically and empirically) guaranteed 😀😀😀
Core idea: 🚨Do not update if you are not sure
👨‍💻https://t.co/hrbtWQO53X
🤗https://t.co/xCLOEXSVX5
📚https://t.co/7JdekGRMOO

@cranialxix @lqiang67 @Tim38463182

[Kaizhao Liang @KyleLiang5]

I did it boys! 💥💥💥Native search into the thinking process. 💥💥💥
👋👋 goodbye agentic frameworks
We are so early!
#DeepSeek #DeepResearch #AgenticAI

[Bo Liu @cranialxix]

How to design State Space Models (SSM) from principles? We propose to view SSM's recurrence as the per-step closed-form solution to an online learning problem. To this end, we present Longhorn, a novel SSM that achieves 1.8x better sampling efficiency against Mamba.

[Kaizhao Liang @KyleLiang5]

😲 crazy
https://t.co/uk0UWYXGkX

[Xingchao Liu @XingchaoL]

😀For those curious about the math of rectified flow, the paper is here: https://t.co/HGI1EbRZKG

[Xingchao Liu @XingchaoL]

We present PeRFlow (Piecewise Rectified Flow):

Fast sampling.
Efficient training.
Transferrable to various SD-based models/methods.
Fully Open-Sourced.

➡️Website: https://t.co/X2fKqbs9fS
➡️Github: https://t.co/pnjDZzEazr

Screenshot for fast text-to-multiview with PeRFlow:

[Xingchao Liu @XingchaoL]

#SD3 use flow! In ICLR 2023 submission, there were rectified flow (https://t.co/i084ocroHk), flow matching (https://t.co/kyTDcjzXSa), stochastic interpolant (https://t.co/VbzQCTeirj), alpha-blending (https://t.co/chzNj9Rb2l), action matching (https://t.co/BXxIrnpHjq). Flow!🚀

[Xingchao Liu @XingchaoL]

We're thrilled to release the codes and pre-trained models of InstaFlow. It pushes Stable Diffusion to one-step generation, and now we find it compatible with pre-trained ControlNets.

Github: https://t.co/Cp6MXvu2Zg

Images generated in One-Step:

[Alireza Makhzani @AliMakhzani]

Introducing “Wasserstein Lagrangian Flows”: A novel computational approach for solving Optimal Transport and its variants.

Paper: https://t.co/qq7fVDBtxk
Led by @k_neklyudov and @brekelmaniac
With: @AlexanderTong7 @lazar_atan @lqiang67

The solution of Optimal Transport (OT) and many of its variants, such as Schrödinger bridge, unbalanced OT, or OT with physical constraints, can be formulated as a curve on the density manifold that minimizes a particular Lagrangian action functional objective.

We propose a unified computational framework for solving a very general class of action functional minimization problems, which also solves OT and its variants. We derive the dual objective of the Lagrangian on the cotangent space of the density manifold, resulting in a min-max optimization. It turns out the inner optimization, that evaluates the action functional, is a generalized “action-matching” objective, which we have previously shown to be tractable with a neural network parameterization: https://t.co/N0jsXhvzQI. The outer optimization modifies the curve to minimize the action functional, subject to endpoint or multi-marginal constraints, using the reparameterization trick.

The flexibility of the choice of Lagrangian in our framework gives us a principled way to incorporate different prior information in our problem, which finds many applications in AI4Sicence and generative modeling. We evaluate our method in trajectory inference problems in biological systems; and also show its applications in enabling fast one-step image generation.

See Kirill’s thread for more details👇

[Kirill Neklyudov @k_neklyudov]

I'm excited to share that our paper got accepted for #NeurIPS2023 as a spotlight!
@JannesNys @lucaathiede @carrasqu @lqiang67 @wellingmax @AliMakhzani