Chenwei Xu

Statistics & Data Science PhD Student@Northwestern

[Ching-Yuen Huang @Michael_Huang_W]

🎉 OASIS is accepted to NeurIPS 2026!
🔗A follow-up to Kimi Attention Residuals @Kimi_Moonshot
⚠️ We find that the extra depth-wise Softmax in Attention Residuals can amplify attention sinks, activation outliers, and low-bit quantization errors.
💡 OASIS adds explicit null routes to both token and depth routing, then feeds token-level null signals back into depth routing.
📈 Across multiple models:
↓95.9% kurtosis / ↓82.0% W8A8 PPL / ↑42.1% W4A4 GSM8K
📚 On 12K context, OASIS also substantially recovers the performance drop of AttnResidual on RULER and LongBench.
✨ More robust Attention Residuals for low-bit inference and long context.
Thanks to all the collaborators across 8 institutes! @robinluo1997 @HHarryD @Michael_Huang_W @NorthwesternU @illinoistech @RutgersU @UMich @UCLA @UCSD @TAMU @northwesterncs @iitcsdept @RutgersCS

Paper Link: https://t.co/4WLtfVv3K2
Project Website: https://t.co/S3m8Y4PcV3

[Kevin H. Zhao @ggkhzhao]

(1/N) autoresearch 🤝 weather forecasting - a thread 🧵

Can an automatic research loop improve a real weather dynamical core by making physics-informed changes? TBH we weren’t expecting much, but the early results were surprising enough for us to share:

[Chenwei Xu @ChenweiXu]

Very thrilled to announce our paper “Towards Sparse Video Understanding and Reasoning” has been accepted to CVPR 2026!

Please feel free to check our paper and contact us if you have any comments or questions!

Paper: https://t.co/aom8tdLFp8

#CVPR2026 #VLM #EfficientAI

[Biology+AI Daily @BiologyAIDaily]

AlignAb: Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies

1. AlignAb introduces a cutting-edge framework for designing nature-like antibodies, focusing on co-designing both sequence and structure. It achieves Pareto-optimal energy alignment to optimize antibody binding functionality and structural rationality.

2. The three-stage framework includes pre-training a language model with millions of antibody sequences, transferring representations to a diffusion model, and aligning designs using multi-objective energy metrics.

3. A key innovation is Pareto-Optimal Energy Alignment (POEA), which balances conflicting energy preferences such as minimizing repulsion and maximizing attraction at antigen-binding sites. This ensures more realistic antibody designs.

4. AlignAb incorporates iterative online exploration with temperature scaling, leveraging real-time model outputs to enhance diversity and consistency in antibody generation, addressing challenges like mode collapse.

5. Experimental results highlight significant improvements in metrics like CDR Etotal and CDR-Ag ∆G, outperforming state-of-the-art methods such as DiffAb and ABGNN. AlignAb bridges the gap between in silico and in vitro antibody design.

6. By extending Direct Preference Optimization (DPO), AlignAb leverages ground-truth reward models and introduces multi-objective alignment, marking a departure from traditional single-metric optimization in generative models.

7. The model's ability to generate antibodies with closer energy profiles to natural ones establishes its potential for applications in therapeutic antibody development and rational vaccine design.

@ChenweiXu
📜Paper: https://t.co/Kux4DUXlhP
#AntibodyDesign #AI #Bioinformatics #ProteinEngineering #AlignAb

[Stat.ML Papers @StatMLPapers]

BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model https://t.co/r8Ge6KgER9

[Chenwei Xu @ChenweiXu]

On Sparse Modern Hopfield Model #NeurIPS2023
https://t.co/WQwfpYyXqs

[Stat.ML Papers @StatMLPapers]

Feature Programming for Multivariate Time Series Prediction. (arXiv:2306.06252v1 [cs.LG]) https://t.co/gX13j48Uum

[OSUCtrAutoRsrch @OSUCtrAutoRsrch]

A big congrats to the @ohiostate student team who placed 2nd in the @SAEIntl inaugural AI Mini-Challenge! https://t.co/1UxNfg3U2W