Kaili Huang

IR / RL / Multi-modality. Senior MLE @Apple; ex @MicrosoftAI; MSCS @Stanford; BS @Tsinghua_Uni

[Kaili Huang @KailiHG]

Why is opus 4.8 so much slower than 4.7?

[Kaili Huang @KailiHG]

I’ve recently started a new journey as a Senior Machine Learning Engineer at @Apple.

As I step into this exciting new chapter, I want to take a moment to sincerely thank everyone I’ve met along the way - mentors, teammates, and friends who’ve inspired and supported me with their wisdom, kindness, and encouragement.

I'm especially grateful for my time as an Applied Scientist at @MicrosoftAI, where I had the opportunity to work on cutting-edge GenAI, retrieval, and multi-modal models, collaborate with brilliant minds, and grow both technically and professionally. It was a place where I learned not just how to build impactful systems, but also how to think deeply and lead with curiosity.

Now, I’m looking forward to bringing that same passion and spirit to the work ahead at Apple, building thoughtful, scalable ML systems that enrich everyday life through innovation and push the boundaries of what's possible.

To everyone who has been part of my journey so far, thank you. And to those I haven’t met yet, I'm excited for the paths we may cross.

Onward to new adventures! (Or as one friendly NPC would say: "Off on another adventure, are we?!" ✨)

[Kaili Huang @KailiHG]

I’m attending #NeurIPS2025 from today to Friday. Looking forward to meeting and chatting with friends here!

[Kaili Huang @KailiHG]

@sama Small-but-important update!

[Sam Altman @sama]

Small-but-happy win:

If you tell ChatGPT not to use em-dashes in your custom instructions, it finally does what it's supposed to do!

[Kaili Huang @KailiHG]

@sama My best friend.

[Sumit @_reachsumit]

ColBERT-serve: Efficient Multi-Stage Memory-Mapped Scoring

@KailiHG et al. introduce a memory-mapping strategy for ColBERT retrieval models that reduces RAM usage by 90% while maintaining low query latency.

📝https://t.co/e9PTyRcLJB
👨🏽‍💻https://t.co/XRBE8UYXI6

[Kaili Huang @KailiHG]

@potemkin_ai Yes! Check out this GitHub repo: https://t.co/Uzw8Ko67w8 🚀🚀🚀

[Kaili Huang @KailiHG]

🚀Excited to share that our paper "ColBERT-serve: Efficient Multi-Stage Memory-Mapped Scoring" was accepted to #ECIR2025. Huge thanks to my amazing collaborators! Looking forward to insightful discussions—let’s connect! 🇮🇹🔍 #NLP #InformationRetrieval

[Kaili Huang @KailiHG]

If you're going too and would like to chat—about research, work, or just to say hi—feel free to DM me! Would love to connect. 😊

[Weizhu Chen @WeizhuChen]

Excited to share our paper "Not All Tokens Are What You Need for Pretraining" received the #NeurIPS2024 Best Paper Runner-up award. I’ll be attending the conference from Wed to Sat. Feel free to reach out if you'd like to connect or attend our talk.
https://t.co/81YTbRoppy

[DeepSpeed @DeepSpeedAI]

Introducing Domino: a novel zero-cost communication tensor parallelism (TP) training engine for both single node and multi-node settings.

- Near-complete communication hiding
- Novel multi-node scalable TP solution

Blog: https://t.co/08bPanyr9M