[1/n] Does AlphaFold3 "know" biophysics and the physics of protein folding? Are protein language models (pLMs) learning coevolutionary patterns? You can try to guess the answer to these questions using mechanistic interpretability. But the thing is, more often than not, we know beforehand what we want/expect our model to learn, especially in very well-studied domains like Biology. So, instead of playing the guessing game, we trained a model that explicitly learns different biophysical and biochemical concepts that biologists care about.
In our concept bottleneck protein language model paper, we show that we can train the model with billions of parameters, with interpretability constraints, without performance degradation.🧵
Joint work with @nc_frey@tuomasoi@amywang01@juliusadml@samuel_stanton_@taylor_joren@0x00B1@hcorrada@kchonyc and Joseph Kleinhenz at @PrescientDesign
@guardiannews Or crack down on / double the postage costs for junk, circulars, unnecessary bits of paper from the bank and bills companies? Almost 100% of the mail I get goes straight in the bin.
3/ Two colleagues can’t reproduce. It's weird and confusing and I'm scared. I’m using an @nvidia Tesla T4 with Cuda 12.0 on @awscloud. Not sure where I would file a bug or if it even is a bug and not just a weird mess!
@ncarleson @PhilippBayer@actually_alan Hi Nick! Good idea. Alan probably needs an update.
In the meantime you can use Alanis Morissette https://t.co/zajJ1QAkES, which is a variant of the original Alan one-liner.
Thanks @carlyraejepsen for spending 2,713 minutes with me in 2022. I couldn’t stop listening to Cut To The Feeling. #SpotifyWrapped https://t.co/lNWzP1600C