DeWeese Lab

the lab of Michael DeWeese at UC Berkeley | neuroscience, neural nets, and nonequilibrium stat mech | | run by students

[DeWeese Lab @DeWeeseLab]

Very excited to share our new paper out in PRL today (as an Editors’ Suggestion)! We unify thermodynamic geometry 🔥 and optimal transport 🥖 providing a coherent picture for optimal minimal-work protocols for arbitrary driving strengths 💪 and speeds ⏱️ ! https://t.co/FHm69Ts7yv

[DeWeese Lab @DeWeeseLab]

@benkuznetsspeck A nice figure from the manuscript (-:

[DeWeese Lab @DeWeeseLab]

We are pleased to see our paper with @benkuznetsspeck freshly published on PRE! We derive a novel time-asymmetric fluctuation theorem that provides the theoretical foundation for stochastic normalizing flows, useful for efficient free-energy estimation! https://t.co/sAhV6hvtEI

[DeWeese Lab @DeWeeseLab]

@benkuznetsspeck @PhysRevE A nice figure from the manuscript :-)

[DeWeese Lab @DeWeeseLab]

@indy_slug @_dsevero @liotier *enhance*

[Bored Yann LeCun @boredyannlecun]

Found a kindred spirit at #icml2023 #feelthelearn

[DeWeese Lab @DeWeeseLab]

The upshot: representations are build one dimension at a time in a process of gradual rank increase. This process -- which can be strikingly observed even with e.g. SimCLR trained off-the-shelf! -- is quite general, and an almost-inevitable consequence of model symmetry.

[DeWeese Lab @DeWeeseLab]

Interested in representation learning or self-supervised learning (SSL)? Check out this work led by lab member Jamie Simon, which gives the first compelling scientific picture of the training process of modern SSL methods like SimCLR! Appearing at ICML '23.

[Imbue @imbue_ai]

Self-supervised learning lets neural nets learn from unlabeled data. Yet despite its success w/ models like @Midjourney, how image representations are learned has been poorly understood.

In a @ICMLConf paper, we share the first scientific picture of the SSL training process 👇

[Josh Albrecht @joshalbrecht]

Happy to see Jamie's research about the stepwise nature of SSL on the BAIR blog: https://t.co/oVkaxCQt2Y

If you're interested in learning more, stop by and say hi to us at ICML next week!

[DeWeese Lab @DeWeeseLab]

Lab member @dhruvakarkada just revamped the NTK wikipedia entry to make it more accessible to technical non-experts. cool gif inside -- check it out!
https://t.co/MFkU8Ht31s

[DeWeese Lab @DeWeeseLab]

We're happy to announce that lab member Jamie Simon is a P̶e̶r̶m̶a̶n̶e̶n̶t̶ ̶I̶n̶t̶e̶r̶n̶ Research Fellow with the startup and a coauthor on the Avalon paper!

[DeWeese Lab @DeWeeseLab]

Check out @genintelligent, a new startup aiming to develop fundamental understanding of AI, and their Avalon environment for the development and testing of generally capable RL agents!

[Kanjun 🐙 @kanjun]

Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why?

We're launching @genintelligent & open sourcing our research environment, Avalon, so as a community we can answer these questions: https://t.co/dNs0TpQeh3

[Kanjun 🐙 @kanjun]

In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"

2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"

Our answer: https://t.co/QnxTbfemTA

[DeWeese Lab @DeWeeseLab]

Thanks to several folks for input, including @PreetumNakkiran and @dbeagleholeCS.

[DeWeese Lab @DeWeeseLab]

Seen the recent papers demonstrating that neural nets' learned functions are very close to kernel regression (KR) with the net's *post-training empirical NTK*? What properties should we expect in such data-adapted kernels? Our new arXiv note suggests some:
https://t.co/XsDedeTY7x

[DeWeese Lab @DeWeeseLab]

galvanizing thinking about *adaptive* kernel methods. If we understood how a neural net's NTK really adapted to data, this would go a *long* way towards clarifying nets. This note is a first little step towards clarifying how we might think about such objects.

[DeWeese Lab @DeWeeseLab]

just updated https://t.co/5KmNrmY0YG! check it out for a view of new research and new lab members.

[Berkeley AI Research @berkeley_ai]

How can we design neural networks in a principled way? New work from @DeWeeseLab @berkeley_ai explores a theoretically-motivated paradigm for doing just that for dense networks! Read about it here: https://t.co/jPmMz9Ab9R