William Tong

Applied math PhD student @Harvard. Neural networks and reasoning.

[William Tong @wl_tong]

Just how useful are reasoning traces?

Together with Ege Cakar and @CPehlevan, we're thrilled to announce a new preprint!

https://t.co/Nshkgalh95

[1/7]

[William Tong @wl_tong]

Many thanks to my extremely talented mentee Ege Cakar, exceptionally supportive advisor @CPehlevan, members of the Pehlevan Group @Harvard and the @KempnerInst! Special thanks for the generous support from @patrickshafto and the @DARPA AIQ program❤️

[7/7]

[William Tong @wl_tong]

Just how useful are reasoning traces?

Together with Ege Cakar and @CPehlevan, we're thrilled to announce a new preprint!

https://t.co/Nshkgalh95

[1/7]

[William Tong @wl_tong]

Reasoning models do not always dominate their "non-reasoning" counterparts, and much depends on task topology.

For more frumentaceous findings and a deeper theoretical analysis, please see our preprint:

https://t.co/Nshkgalh95

[6/7]

[William Tong @wl_tong]

Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst!

If you're at CCN 2025 (@CogCompNeuro) next week, please swing by my poster on Tuesday and say hello!

🧵[5/5]

[William Tong @wl_tong]

@CPehlevan and I are thrilled to announce our next paper! We explore equality reasoning in neural networks, and the impact of learning richness.

https://t.co/txKz6QvvL7

🧵[1/5]

[William Tong @wl_tong]

Our conclusions also hold for visual reasoning tasks. In this example, the model must distinguish whether the pentominoes are the same shape or different. Richer models generalize faster from fewer training shapes.

🧵[4/5]

[Benjamin S Ruben @BenjaminSRuben]

Check out my latest work, studying a fundamental trade-off in machine learning: should you train a single large model, or an ensemble of smaller models? A huge thanks to my advisor @CPehlevan and collaborators @wl_tong and @hamzatchaudhry.

https://t.co/CIetU02CtS

[William Tong @wl_tong]

@uzpg_ Yep! https://t.co/6ZoJZMifkf

[William Tong @wl_tong]

MLPs learn in-context.

For example, MLPs (and the closely related MLP-Mixer) attain competitive loss with Transformers on an in-context classification task, comparing by compute.

🧵[1/n]

[William Tong @wl_tong]

@BlackHC Thanks, and great point! The context length is indeed fixed in MLPs. One workaround used in Mixers is to fix a large maximum context length (which is also employed for Transformers with certain positional encoding schemes)

[William Tong @wl_tong]

Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst!

Please feel free to reach out with questions or comments!

🧵[5/n]

[William Tong @wl_tong]

In this Same-Different task, the model receives two, one-hot encoded input tokens, and must output whether they are the same or different.

With a large enough training vocabulary, MLPs generalize to *unseen* tokens.

🧵[4/n]

[William Tong @wl_tong]

We also find that MLPs excel at relational reasoning.

In this example Oddball task, an MLP generalizes substantially better than Transformers to out-of-distribution distances.

🧵[3/n]

[William Tong @wl_tong]

@CPehlevan and I are thrilled to announce a new preprint! We explore the surprising capacity for MLP models to learn in-context.

https://t.co/ScWQGKgbLd

🧵[2/n]