My new article "Toward a science of intelligence: unifying physics, neuroscience and AI" https://t.co/9jVmJzg1BW
published in the Daedelus journal of @americanacad
Its part of a special issue on AI+Science with many amazing contributors lead by James Manyika https://t.co/3Z7aer186F
Had a great week at the kick-off workshop for @SimonsFdn Collaboration on the Physics of Learning and Neural Computation.
So many brilliant people working on fascinating problems with a genuine passion for science. Thanks @SuryaGanguli and all the PIs for organizing!
We have 14 survey lectures for our @SimonsFdn Collaboration on the Physics of Learning and Neural Computation! All videos available at: https://t.co/MLnVYY6Fhh
Here is the list:
@zdeborova: Attention-based models and how to solve them using tools from quadratic networks and matrix denoising
@KempeLab: Recent lessons from LLM reasoning
@MBarkeshli: Sharpness dynamics in neural network training
@KrzakalaF: How Do Neural Networks Learn Simple Functions with Gradient Descent?
Michael Douglas: Mathematics, Economics and AI
Yuhai Tu: Towards a Physics-based Theoretical Foundation for Deep Learning: Stochastic Learning Dynamics and Generalization
@SuryaGanguli: An analytic theory of creativity for convolutional diffusion models
Eva Silverstein: Hamiltonian dynamics for stabilizing neural simulation-based inference
@adnarim066: Generation with Unified Diffusion
Bernd Rosenow: Random matrix analysis of neural networks: distinguishing noise from learned information
@jhhalverson Nerual networks and conformal field theory
@KempeLab Synthetic data: friend or foe in the age of scaling
@WyartMatthieu Learning hierarchical representations with deep architectures
@CPehlevan Mean-field theory of deep network learning dynamics and applications to neural scaling laws
1/ StatPhys helps analog computing! We investigate the interplay between geometry and dynamics of the coherent Ising machine against problems with hidden and planted solutions https://t.co/qgHStiPK87 w/ @adithya_sriram@atsushi_y1230@SuryaGanguli@hmabuchi 🧵
9/ Overall, we found intriguing relations between the high dimensional geometry and dynamics in analog machines, and achieve a mechanistic understanding of success and failure modes of this device. Hopefully this work contains lots of inspirations to improve the CIM even further!
The return of the physicists: "CMT-Benchmark: A benchmark for condensed matter theory built by expert researchers." https://t.co/lqaiAtozy1 A set of hard physics problems few AIs can solve. Avg performance across 17 models is 11%. Problems range across topics like:
Hartree-Fock (HF)
Exact Diagonalization (ED)
Density Matrix Renormalization Group (DMRG)
Quantum Monte Carlo (QMC)
Variational Monte Carlo (VMC)
Projected Entangled Pair States (PEPS)
Statistical Mechanics (SM)
And more (OTHER).
Table shows performance (percent correct) of models across problem classes. In many cases 0% correct!
General lessons on how to derive hard problems for AI are in the paper.
Was a fun collaboration with 10 condensed matter theory labs across the world.