Zhimeng Guo

PhD @ Penn State. Intern @ Bytedance Seed. Pretraining, Long context and agent.

[Andrew Gordon Wilson @andrewgwils]

In cooking, execution is more important than the dish itself, even for simple dishes. Hummus can be great or terrible. The same is true of scientific ideas. Almost nothing works as we wish at first. Persistence, high standards, and attention to detail make all the difference.

[Morty @0xMortyx]

Andrej Karpathy Anthropic Head of Technical Staff:

"Multi-task learning has a team problem nobody talks about."

Karpathy's team runs 100 subtasks inside one neural network

In this 15-minute talk, Karpathy breaks down the full multi-task stack:

architecture tradeoffs + loss balancing + data engines + team workflow.

Worth more than any $500 ML bootcamp.

Watch it & then read the full article below

[Stefano Ermon @StefanoErmon]

LLM-assisted search in verifiable domains is incredibly exciting right now. The model matters, but so does the algorithmic harness used to explore the search space and iterate toward better solutions.

We’re excited to describe a new search algorithm that makes this exploration process more effective, leading to the results below with open-source models:

✨ Mathematics: new state-of-the-art constructions for the Erdős Minimum Overlap Problem

⚛️Quantum computing: improved quantum circuit compilation, reducing SWAP overhead by 24.5% on IBM Q20.

⚡️ AI infrastructure: designed a highly efficient TriMul Triton kernel, improving on prior human- and AI-designed implementations.

More details in the blog: https://t.co/h4lMnYHM4I

Great collaboration by WILL, @Stanford, @PKU1898, @Tsinghua_Uni, and @HKUSTGuangzhou.

[Vatsal Baherwani @vatsalbaherwani]

Scaling laws predict an LLM's pretraining loss, but not its capabilities. Abilities like in-context learning emerge abruptly and only past a certain scale. Our new paper traces this to one bottleneck: learning which tokens attention should focus on. 🧵https://t.co/ja0wc8aK2e

[Zhimeng Guo @TimeLoverc]

@ChenhaoTan Hi Professor, I actually have a rough implementation of this! The logic is simple: you send ideas (random or serious) to a Slack bot, and it attempts to build them overnight. It's still very experimental, but it’s a start... (link:https://t.co/19SsHdIZxI)

[Zhimeng Guo @TimeLoverc]

@xdotli @josancamon19 Thanks for the shoutout @xdotli! 🙌 We are indeed focusing on making agents more robust for these long-horizon tasks.

[Huaisheng Zhu @huaiszhu]

🚀 New Paper Alert!

🧠 Simple Denoising Diffusion Language Models (SDDLMs)
We simplify the complex ELBO objectives in Uniform-State Diffusion Models with a simple denoising loss, making training more scalable — while matching or surpassing baseline generation quality.
(1/2)

[Jascha Sohl-Dickstein @jaschasd]

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.

[Noam Shazeer @NoamShazeer]

https://t.co/abQu8pN4WJ
Check out my first blog post.

[Samuel Albanie 🇬🇧 @SamuelAlbanie]

TLDR: Unsupervised knowledge discovery in LLMs is hard

Intriguing theoretical and empirical results from @seb_far et al.

Paper: https://t.co/eIoBpKtfp2

And for those who enjoy video summaries: https://t.co/sGF7XwDzrl

[Max Welling @wellingmax]

I have noticed that the “review process” is very similar to mcmc: the further away from the current state you propose the less likely it is to be accepted”.

[Itai Yanai @ItaiYanai]

Out today in @NatureBiotech! PhD students & postdocs: there's a trick – overlooked and underused – for having new ideas in the creative process. Talk science 1 on 1 with a science buddy that you trust and think of it as an improvisation. https://t.co/Xw7VpKHot2…
@MartinJLercher

[Zhimeng Guo @TimeLoverc]

@sirbayes Amazing work! Congratulations!

[Zhimeng Guo @TimeLoverc]

Yay! I’m Tweeting from the API! 🎉

[Zhimeng Guo @TimeLoverc]

Begin to use twitter.