@zdeborova@zdeborova , you made a non-trivial statement. Non-memorization means with all other training conditions equal it learns a best approximation of a true distribution, which gives a best generalization. So, you are saying it is not? Do you have an example?
@andrewgwils DL theory that explains could be useful but what we really need is something more practical, that excludes guessing from finding new optimal architectures.
Insightful blog post from @ShunyuYao12 about what's needed for next level of AI progress. He's been driving AI progress since his work @PrincetonPLI including SWE-Bench.
Related but somewhat divergent view from @karpathy who argues (correctly in my opinion) that we need to go beyond current simplistic ideas in RL. https://t.co/qqvpVCSGh2
🧵 What if two images have the same local parts but represent different global shapes purely through part arrangement? Humans can spot the difference instantly! The question is can vision models do the same?
1/15
Transition Matching (TM), a novel discrete-time, continuous-state generative paradigm that unifies and advances both diffusion/flow models and continuous AR generation:
Transition Matching: Scalable and Flexible Generative Modeling
"This paper introduces Transition Matching (TM), a novel discrete-time, continuous-state generative paradigm that unifies and advances both diffusion/flow models and continuous AR generation. TM decomposes complex generation tasks into simpler Markov transitions, allowing for expressive non-deterministic probability transition kernels and arbitrary non-continuous supervision processes, thereby unlocking new flexible design avenues."
Integration Flow Models
"Integration Flow is the first model with a unified structure to estimate ODE-based generative models and the first to show the exact straightness of 1-Rectified Flow without reflow."
The most advanced AI systems will likely exist first within the companies that developed them, with little opportunity to collectively prepare for their impacts. This dynamic creates unique risks highlighted in this recent paper by @apolloaievals and should become a growing priority for AI policy.
@andrewgwils Simple models of the world and wish for quick rewards is a road to disaster. That's why principles are more important than models. Principles are wisdom derived from ages of human history. We have to stick to it to make good and make it right.
@docmilanfar That's so much fun: having a bottle that's never full. But what it means for images? Are there any integral topological invariants on image space?