Pengxiang Li

PhD student @HongKongPolyU

[Pengxiang Li @pixeli99]

With just 7M params:
• 97.9% on Sudoku-Extreme
• 13.6% pass@2 on ARC-AGI-2
• Still improving beyond 20,000 test-time loops

More loops → better reasoning.
💻 https://t.co/9RKctbtMcu

w/ @pumpkinnnnne @luu_yinn @Shiwei_Liu66

[Pengxiang Li @pixeli99]

Looped Transformers should get better the more they loop. In practice, more loops often make them worse. We call this latent overthinking.

We found why, and how to fix it. Introducing InfiLoop 🧵
📄 https://t.co/vHheAOmTvq

[Pengxiang Li @pixeli99]

Our fix: loop-native residual connections.

Instead of only the latest state, InfiLoop keeps a running summary of past states, weighted by content + learned decay—downweighting noisy updates while memory stays constant as loops grow.