Lexiao Lai

Assistant Professor @ HKU Math

[Lexiao Lai @lailexiao]

Our stochastic range matches results in smooth counterparts.

• Smooth SGD — Dereich & Kassing: https://t.co/tGhKtEPLCT

• Smooth SGDM — @JunwenQiu, Ma & Milzarek: https://t.co/YJg9Xv0k6i

Comments welcome!

3/3

[Lexiao Lai @lailexiao]

Do bounded momentum and stochastic subgradient methods converge to a critical point? For locally Lipschitz definable objectives, yes—not just subsequences, under step sizes depending on objective geometry.

New preprint w. @MingzhiS13149: https://t.co/7wznnOJ74N 1/3

[Lexiao Lai @lailexiao]

Why is the momentum range tight? For step sizes of order k^{-1/2}, we construct an example where the subgradient method (zero momentum) admits a bounded nonconvergent realization, even under the more strict active setting. 2/3

[Lexiao Lai @lailexiao]

Update to our MCSD work: the framework now extends beyond smooth manifolds to structured nonsmooth feasible sets—including the low-rank variety and spectral-norm sphere—with a new hybrid MCSD–PGD method and convergence guarantee.

https://t.co/X85GIU8kTl

[Lexiao Lai @lailexiao]

New preprint: Manifold Constrained Steepest Descent (MCSD).

MCSD is intentionally minimal: each iteration uses just
• a norm-based steepest direction via an LMO, and
• a projection back to the manifold.

Single loop. Guaranteed to converge.

Paper:
https://t.co/p1tGBXN2eG

[The University of Hong Kong @HKUniversity]

The University of Hong Kong (HKU) welcomes Professor Ngô Bảo Châu, recipient of the 2010 Fields Medal, to join the Department of Mathematics under the Faculty of Science, commencing June 2026. Professor Ngô earned global recognition for his groundbreaking proof of the fundamental lemma in the Langlands Programme, a monumental achievement that solved a three-decade-old mathematical challenge and was named one of Time magazine's Top 10 Scientific Discoveries of 2009.

The Langlands Programme, proposed by mathematician Robert Langlands in 1967, stands as one of mathematics' most ambitious theoretical frameworks, creating vital connections between number theory and group theory. For three decades, its fundamental lemma remained unproven until Professor Ngô's ingenious solution in 2009, which not only validated years of mathematical work but also opened new pathways within this profound theoretical structure.

In 2010, Professor Ngô made history by becoming the first Vietnamese mathematician awarded the Fields Medal – mathematics' highest honour for researchers under 40. Along with the Abel Prize, these awards are widely regarded as the discipline's equivalent of the Nobel Prize. His transformative contributions to number theory and representation theory have had far-reaching implications across mathematical physics. His distinguished honours include the Clay Research Award (2004), Oberwolfach Prize (2007), Sophie Germain Prize (2007), and election as a Member of the American Academy of Arts and Sciences (2012) and a Foreign Member of the French Academy of Sciences (2016).

read more: https://t.co/x8H6p8VGRA

#HKU #香港大學 #UniversityofHongKong

[Lexiao Lai @lailexiao]

@nilayparikh Thank you for your interest! Very good question, in our work we consider orthogonal constrained CNNs and a new LoRA approached called StelLA. For modern applications, I think there are also some quantization tasks where people find orthogonality can be helpful.

[Lexiao Lai @lailexiao]

New preprint: Manifold Constrained Steepest Descent (MCSD).

MCSD is intentionally minimal: each iteration uses just
• a norm-based steepest direction via an LMO, and
• a projection back to the manifold.

Single loop. Guaranteed to converge.

Paper:
https://t.co/p1tGBXN2eG

[Lexiao Lai @lailexiao]

@tonysilveti Thank you for sharing our work!😆

[Lexiao Lai @lailexiao]

Empirically: on PCA, SPEL is competitive in iterations and much lighter than existing Manifold Muon (MM) baselines in wallclock time. Also competitive on orthogonal CNNs and LLM adapters, more experiments in the paper.

[Lexiao Lai @lailexiao]

SPEL: MCSD with spectral norm on the Stiefel manifold. An analogue of Muon on Stiefel manifold, using efficient implements of msign. Unlike many manifold optimizers whose descent directions are on the tangent space, SPEL allows an ambient descent direction before projection.

[Lexiao Lai @lailexiao]

Our contribution:
1. We bound the diameter of subgradient sequences using tools from o-minimal geometry.
2. As a corollary, for tame objectives, bounded subgradient sequences converge to a unique critical limit when step sizes are ~1/k.
Comments and suggestions welcome!
3/3

[Lexiao Lai @lailexiao]

Do iterates of the subgradient method converge for nonconvex, nonsmooth objectives?

Yes, for tame/definable functions, covering most ML/AI training objectives. Check out our new preprint https://t.co/gFVKBVAQ2E (w. my PhD student @MingzhiS13149) #nonconvex #optimization

1/3

[Lexiao Lai @lailexiao]

Prior arts on subgradient sequences (iterates):
Alber et al, 98: For convex objectives, sequences converge to minimizers.
@damekdavis, Drusvyatskiy, @jasondeanlee, & @ShamKakade6, 20: For tame objectives, bounded sequences' limit points (might not be unique) are critical.

2/3