Fu-Ming Guo

Chief Architect AI/ML @ Visa. Opinions are my own.

[Fu-Ming Guo @FumingGuo]

@shawnyinmr 车都可以当抵押品，为什么算力不行呢？

[Fu-Ming Guo @FumingGuo]

@pmddomingos Exactly! Human brain and the whole human civilizations recorded by language

[Fu-Ming Guo @FumingGuo]

@hugo_larochelle @Mila_Quebec Congratulations!🎉

[Zeyuan Allen-Zhu, Sc.D. @ZeyuanAllenZhu]

❓Can I collect some feedbacks: Is fully open-source research necessary?
Earlier, I released a family of 1-8B models (open data, code, weights): beating Llama3-8B with <10% pretrain time, beating most (all?) open-data models of this scale.

🔓No shortcuts: 10+ legal debates for using open data, 10+ more for weights, months of blockers, endless nights scavenging GPUs.

📈All to provide a strong, reproducible baseline — one I believed critical for understanding the physics of LLMs. Next planned was GLA + Canon, outperforming all modern linear models in tests.

Yet attention was low.

❓Should I close-source to save time and focus on pure research? Honest feedback appreciated.

[Fu-Ming Guo @FumingGuo]

Congratulations! Well deserved!

[ICLR @iclr_conf]

Test of Time Winner

Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba

Adam revolutionized neural network training, enabling significantly faster convergence and more stable training across a wide variety of architectures and tasks.

[Fu-Ming Guo @FumingGuo]

Congratulations! Well deserved!

[🇺🇦 Dzmitry Bahdanau @DBahdanau]

@iclr_conf many many many thanks to @kchonyc and @Yoshua_Bengio for enabling the wildest ever start of my research career

2014 was a very special time to do deep learning, a commit that changes 50 lines of code could give you a ToT award 10 years later 😲

[Fu-Ming Guo @FumingGuo]

@srush_nlp Same here. Email is also slow. Download a kindle app on iPhone, open the arxiv link pdf file version on iPhone, share to the kindle app through Safari sharing button, almost immediately appeared on Kindle Scribe. Only issue to improve is that scribe size larger would be better

[Fu-Ming Guo @FumingGuo]

@Wenxuan_Zhou I think your suggestion is a better approach based on the conclusions in recent amazing research from @ZeyuanAllenZhu Physics of Language Models

[James Zou @james_y_zou]

🏆Exciting that our Mixture of Agents (MoA) tops the AlpacaEval leaderboard!

We introduce the MoA architecture: layers of diverse LLM agents fuse + improve prev LLMs' outputs. MoA of only open-source LLMs outperforms GPT-4o by 7% on AlpacaEval2 while more cost-efficient🚀1/5

[Chelsea Finn @chelseabfinn]

How can we train full-size humanoid robots?

New paper introducing:
- learned controller for shadowing humans
- imitation learning of demos collected via shadowing

Website with code &amp; videos: https://t.co/uX2aEahPCL

[OpenAI @OpenAI]

We're sharing progress toward understanding the neural activity of language models. We improved methods for training sparse autoencoders at scale, disentangling GPT-4’s internal representations into 16 million features—which often appear to correspond to understandable concepts.

https://t.co/tTRPztmra1

[Furong Huang @furongh]

9/ 🙏 Shoutout to our team and collaborators who made this research possible! 💐 @SOURADIPCHAKR18 @ghosal_suvra @MingYin_0312 @dmanocha @MengdiWang10 @amritsinghbedi3 @furongh #TeamWork

🔗 to paper: https://t.co/V5rvnqBOig

[Hui Guan @guanh01]

MLSys is around the corner. Catch @Azaliamirh talk at the #MLSys2024 Young Professionals Symposium! May 13th in Santa Clara, California. Full agenda here: https://t.co/ZVo1TVf3a6

[Tianqi Chen @tqchenml]

Catch @KurtKeutzer keynote talk at the #MLSys2024 Young Professionals Symposium! May 13th in Santa Clara, California. Full agenda here: https://t.co/9oNuFRTJai

[Fu-Ming Guo @FumingGuo]

@agihippo FlashAttention?

[Fu-Ming Guo @FumingGuo]

@agihippo Interesting guessing game 😄 learning prom your reply ~

[Yi Tay @YiTayML]

It's been a wild ride. Just 20 of us, burning through thousands of H100s over the past months, we're glad to finally share this with the world! 💪

One of the goals we’ve had when starting Reka was to build cool innovative models at the frontier. Reaching GPT-4/Opus level was a personal goal for many of us in the team. Doing it from scratch, on top of starting a company, makes it even more challenging but rewarding. 😁

Core is still improving (not done training!) but we’re happy to ship an early version 🚢. I’ve been vibe-checking it for a bit and it’s a really cool model (especially at multimodal) 😎.

Check out the blogpost, technical report and very non-cherry picked, “in the wild” showcase/demo in the thread below! Core is competitive with true frontier models. It beats Claude3 Opus on multimodal chat and matches GPT4-V on MMMU. Text metrics are competitive too (~83+ MMLU). In my mind, this is our arrival at the frontier. 😎👌🔥

More fun stuff to come in the following weeks! 😋

[Fu-Ming Guo @FumingGuo]

@lupantech @UCLAComSci @kaiwei_chang @uclanlp Congratulations!

[Sasha Rush @srush_nlp]

Graham Neubig - Can we make building with open-source AI as simple as prompting ChatGPT? (@gneubig )

https://t.co/imP5YndsCb

[The Nobel Prize @NobelPrize]

Happy International Women's Day!

We're celebrating women who have changed the world. Here's all of the amazing women who have received the #NobelPrize and their remarkable achievements at the time of the award.

Who are the women who inspire you the most?

#IWD2024