Really interesting paper, one of my favorites I’ve read recently!
Token-level entropy is a common metric used to assess the health of RL training. This paper argues that because token-level entropy only measures diversity within a single response, it does not holistically capture diversity. The model can still respond similarly to different inputs, which is a sign of poor diversity. This type of input-agnostic behavior is referred to as template collapse.
To measure this kind of diversity, a suite of mutual information proxy metrics are proposed that can measure the amount of shared info between responses. These metrics are found to actually correlate more strongly with final performance than entropy, indicating that they may better capture reasoning quality / training health.
1/
When diffusion generates images from text, before an image has objects, how does each noisy token know what it should become?
In our new work, we found that Diffusion Transformers solve spatial-relation prompts using a circuit motif reminiscent of developmental biology: morphogen-like spatial gradients.
At the start of sampling, image tokens are mostly uninformed noise — like an undifferentiated sheet in an embryo. Relation heads then write smooth spatial gradients onto the image canvas, guiding where objects should emerge.
Accepted as a @CVPR 2026 Highlight🌟: https://t.co/oIC6QKwQ3R
Beautiful collaboration with my friends and colleagues @fjxdaisy & Xu Pan!
A 🧵
This one is even better: the right hand side adaptively learns the 4 modes and form a corresponding adaptive partition as the algorithm progresses, the left hand side remains to be a base sampler P.
@31Juno@Arcfunmi You can create anything architecture propose. The real problem is cost. You can increase depth of footing and counter weight to balance momentum create by cantilever.
@sarscov22019 Goldഅനക്കേണ്ട സലറിയിൽ നിന്ന് പോകട്ടെ ബാക്കി കുറേശെ ഗോൾഡ് എടുത്ത് പണയം വച്ചിട്ട് ആണെങ്കിലും കൊടുക്കുക. പലിശ അടക്കാൻ പറ്റുന്നില്ല എങ്കില് മാത്രം ഗോൾഡ് വിക്കുക.
Nb: ഗോൾഡ് കടം/പലിശ ക്ക് വാങ്ങിയവരും ഇപ്പോൾ വൻ ലാഭത്തിലാണ്. ഇന്ത്യൻ രൂപ തയ്നാലും ഡോളർ വീണാലും ഗോള്ഡ് ലാഭത്തിൽ ആയിരിക്കും