Huy Nguyen

Ph.D Candidate in Statistics and Data Sciences at UT Austin @UT_Stats | Prev. Research Intern @ Microsoft GenAI | working on #MachineLearning #MixtureofExperts

[Huy Nguyen @huynm99]

I'm glad to share our #ICLR2025 paper "Statistical Advantages of Perturbing Cosine Router in Mixture of Experts", introducing a novel perturbed cosine router for Mixture of Experts (#MoE) to address the representation collapse and enhance the sample efficiency. [1/n]

[Stat.ML Papers @StatMLPapers]

Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences https://t.co/pUw3Y3H5Hw

[zkpuzzlematt 🧩 @zkmattwyatt]

@chaumian yo wtf this is huge

[chuplung @choopyplug1]

Google DeepMind VP of Research:

"we taught the model to train itself. no human code. no human knowledge. 80% of GitHub is already machine generated. human data ran out"

it creates its own challenges and judges its own answers. models now improve without us. the loop is closed

"we built an entire industry assuming writing code was the hard part. it's now free"

in one year nobody will read code. models generate it. no one looks at it

save this. then read the full article below

[Kempner Institute at Harvard University @KempnerInst]

The application is now open for our #KempnerInstitute Research #Fellowship! Postdocs studying the foundations of #intelligence or applications of #AI are encouraged to apply. Learn more and apply by Oct. 1: https://t.co/AAFQp3NBgn #LLMs #NeuroAI #ML
@ShamKakade6 @blsabatini

[Shahan @shahanmemon]

Are you on academic job market for #Postdoc or #TT Assist/Assoc. prof/Lecturer/#AP/Research Scholar/ #fellowship #jobs for 2026-2027 in #CS/#SocSci/ #InfoSci/Management Science/#Psych & similar? I am maintaining a list (so far ~200 positions).

#academia

https://t.co/jCvb0IQZxI

[Mushtaq Bilal, PhD @MushtaqBilalPhD]

https://t.co/ERrKQgLB3w

[Stat.ML Papers @StatMLPapers]

The Information Geometry of Softmax: Probing and Steering https://t.co/FzHeKNYU0v

[Chanwoo Park @chanwoopark20]

A somewhat well-known property of the softmax function is that it is 1-Lipschitz (from Gao et al 2018).

While thinking about whether this bound is tight, I came across a fun and surprisingly simple result: softmax is in fact 1/2-Lipschitz (Newhouse, Feb 2025). The argument is quite elementary—perhaps too simple for a mathematician—but I nonetheless found it very interesting.

Later, I discovered that Nair (Oct 2025) independently obtained the same 1/2-Lipschitz bound. The difference lies in the choice of the ℓp​ norm, but I am fairly confident that the author was unaware of Newhouse’s earlier work as the author did not cite Newhouse's paper.

Finally—and perhaps most intriguingly—I found a Math StackExchange post from nearly nine years ago showing that softmax is \sqrt{d-1}/d-Lipschitz. Since we assume d≥2, this immediately implies a 1/2-Lipschitz bound as well. Math StackExchange knows everything...

[Wentao Guo @WentaoGuo7]

🚀SonicMoE🚀: a blazingly-fast MoE implementation optimized for NVIDIA Hopper GPUs. SonicMoE reduces activation memory by 45% and is 1.86x faster on H100 than previous SOTA😃

Paper: https://t.co/Xesd3cNcpQ

Work with @MayankMish98, @XinleC295, @istoica05, @tri_dao

[Huy Nguyen @huynm99]

Hi everyone, I’ll be at #NeurIPS this year in San Diego. I’d love to chat about efficient design for Mixture of Experts (#MoE), attention mechanism, and their applications in #LLMs and multimodal learning. Let’s have a coffee if you’re interested in these topics 😃.

[Jeremy Cohen @deepcohen]

Our paper was published last ICLR and is on arXiv here: https://t.co/hlTxjcictN.

We've also created an blog version of the paper, with fun animated optimization trajectories:
https://t.co/avsScLLuDF.

[Huy Nguyen @huynm99]

@_Hiroki11x @MSFTResearch Congrats Hiroki 🎉

[Guangxuan Xiao @Guangxuan_Xiao]

Just wrote a post on my understanding of the statistics behind block sparse attention.

My take is that it works by using the "learned similarity gap," which creates a simple SNR formula connecting retrieval quality with model architecture.

Read more: https://t.co/8jjYLri5rF

[Avi Chawla @_avichawla]

KV caching in LLMs, clearly explained (with visuals):

[Pramod Goyal @goyal__pramod]

A beautiful visual blog, where you can change values, interact, and see what each head does exactly inside the transformer.

[Akshay 🚀 @akshay_pachaar]

Transformer vs. Mixture of Experts in LLMs, clearly explained (with visuals):

[Colaboratory @GoogleColab]

Big news for data science in higher ed! 🚀Colab now offers 1-year Pro subscriptions free of charge for verified US students/faculty, interactive Slideshow Mode for lectures, & an AI toggle per notebook. Enhance teaching & learning in the upcoming academic year! Read all about it here: https://t.co/OCgVqxrHmU. #GoogleColab #EdTech.

[Tri Dao @tri_dao]

State space models and RNNs compress history into a constant size state, while attn has KV cache scaling linearly in seqlen. We can instead start from RNNs and let the state size grow logarithmically with seqlen. Feels like a sweet spot. Also beautiful connection to classical algo like Fenwick tree and hierarchical matrices

[Ji-Ha @Ji_Ha_Kim]

What are the effects of adding epsilon in the denominator in L2 normalization?
Typically, we say it's for numerical stability. But which choice to make and when?

[Kyunghyun Cho @kchonyc]

what do you teach at the first-year graduate course on machine learning, in this era of LLM and large-scale compute? here's my experiment on answering this question: let's teach everything that admits SGD and that is not LLM, and ask students to read old papers.