Yongyi Yang

[Yongyi Yang @YongyiYang7]

DeepSeek's recent mHC (Manifold-Constrained Hyper-Connections) proposes to stabilize residual hyper-connections via 20 Sinkhorn-Knopp (SK) iterations. However, this approach requires heavily customized CUDA kernels and, does not guarantee the quality of stabilization due to approximation errors.

Check our new paper: "mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations". Based on the Birkhoff-von Neumann theorem, we introduce a neat solution that can be realized with standard operators while guaranteeing exact stability.

[Yongyi Yang @YongyiYang7]

@zhanpeng_zhou just ask Claude to translate them

[Yongyi Yang @YongyiYang7]

@carnivalaki 还真是（

[Jianyang Gao @gaoj0017]

The TurboQuant paper (ICLR 2026) contains serious issues in how it describes RaBitQ, including incorrect technical claims and misleading theory/experiment comparisons.

We flagged these issues to the authors before submission. They acknowledged them, but chose not to fix them. The paper was later accepted and widely promoted by Google, reaching tens of millions of views.

We’re speaking up now because once a misleading narrative spreads, it becomes much harder to correct. We’ve written a public comment on openreview (https://t.co/nDVjmNhATM).

We would greatly appreciate your attention and help in sharing it.

[MichiganAI @michigan_AI]

Major milestone in Yongyi Yang's academic journey.🎓

TOMORROW Yongyi Yang will present a dissertation defense on "Structures in Deep Learning: Representations, Learning Dynamics, and Efficient Algorithms."

MARCH 26 @ 3:00pm ET

https://t.co/BWvt7i8owO

[Yongyi Yang @YongyiYang7]

@onimushi 我喜欢你🥰

[Do2mi多多酱 @Do2mi1]

这可太爽了

[Yongyi Yang @YongyiYang7]

📜Read the full paper: https://t.co/7HkD4cUq4V

🥂 Cheers to my wonderful collaborator @gaoj0017, who has been working with me like crazy for the whole past week!

[Yongyi Yang @YongyiYang7]

DeepSeek's recent mHC (Manifold-Constrained Hyper-Connections) proposes to stabilize residual hyper-connections via 20 Sinkhorn-Knopp (SK) iterations. However, this approach requires heavily customized CUDA kernels and, does not guarantee the quality of stabilization due to approximation errors.

Check our new paper: "mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations". Based on the Birkhoff-von Neumann theorem, we introduce a neat solution that can be realized with standard operators while guaranteeing exact stability.

[Yongyi Yang @YongyiYang7]

Why persist in projecting unconstrained matrices onto the constrained convex set? The Birkhoff-von Neumann theorem offers a simple and robust parameterization of doubly stochastic matrices.

By directly using this parameterization, we completely skip the SK iterations, avoid the hassle of specialized kernels, and ensure precise doubly stochasticity by construction.

Experiments confirm that mHC-lite achieves the same (or better) stabilizing effect on training as mHC, while being significantly more efficient and easier to implement.

[Yongyi Yang @YongyiYang7]

📄 Read the full paper: https://t.co/k4fQk8RfHH
Cheers to my amazing co-authors -- Tomaso Poggio, Isaac L. Chuang, and Liu Ziyin @LiuZiyin10

We hope this work helps build a deeper understanding of training, and sparks new bridges between topology, physics, and deep learning.

[Yongyi Yang @YongyiYang7]

check out our new paper "Topological Invariance and Breakdown in Learning." We prove that training exhibits a topological phase transition with a critical learning rate separating topology-preserving and topology-simplifying regimes.. 🧵👇

[Yongyi Yang @YongyiYang7]

🧩This framework is architecture- and optimizer-agnostic.
As long as the model has a permutation invariance on neurons and the optimizer is gradient-based (e.g. SGD, Adam), the theory applies.

This suggests topology as a universal tool for analyzing learning dynamics.

[Yongyi Yang @YongyiYang7]

@YiMaTweets Someone needs to initiate this process

[∀ugust @ModalMetamodel]

Them hoes was tryna figure out if erry compact Hausdorff space wit at least two points and no isolated points got a cardinality of at least 2^ℵ₀. 😸😸😸

[朝潮 @ashashio]

To the end of the world

[Yongyi Yang @YongyiYang7]

(5/5) Read the full paper: https://t.co/vr15isbQTe
Joint work with @weihu_ @Hidenori8Tanaka

[Yongyi Yang @YongyiYang7]

What drives in-context learning in LLMs?

New paper: Provable Low-Frequency Bias of In-Context Learning of Representations.

We show LLMs have a low-frequency bias when learning representations in context, offering a theoretical answer to several previously open questions. 🧵👇

[Yongyi Yang @YongyiYang7]

(4/5) The key idea behind our theory is Double Convergence: hidden states cluster by token identity as context grows (context-wise convergence), and these clusters evolve across layers toward a low-frequency signal over the input's graph structure (layer-wise convergence).