charlie

scientific ai fellow @ notre dame | physics phd @ uiuc

[Brian Hie @BrianHie]

In new work, we lay out a vision for a high-level programming language for generative biology, called Proto.

Proto composes generative and predictive models spanning DNA, RNA, proteins, ligands, and their interactions, which we use to design complex biological functions. 1/n

[charlie @inthisframe]

@Sauers_ codex 5.4 has done similar with "compromise" for me and blamed it on a keyboard layout bug

[charlie @inthisframe]

@eliebakouch very conveniently, the 4.7-4.8 gap is 1.4 and the 4.8-mythos gap is 2.8. see you soon opus 5

[Fern @hi_tysam]

A little over 2 years ago, I solved the SolidGoldMagikarp stability problem.

Today, I am releasing the results of that work as a new technique to regularize training.

More details below.

[charlie @inthisframe]

@seconds_0 https://t.co/P0Ce945zGS

[Just a person @everypoint1]

@nearcyan

[charlie @inthisframe]

@austinc3301 @Shark_Academia The 4.7 model card cites the linked paper and has some explicit discussion of this phenomenon as well

[charlie @inthisframe]

@somewheresy if the titanic could float it wouldn't have hit that iceberg

[charlie @inthisframe]

@AndyMasley The thing I don’t get about the box elder coverage is how both sides seem to be taking the headline numbers at their word. O’Leary’s similar sized wonder valley project seems to be mostly vapor at this point. Where are the tenants or the chips? https://t.co/zFyPbjxHOG

[charlie @inthisframe]

@moultano Do you have any pointers on how to do that/the reasoning behind the theme?

[Dimitris Papailiopoulos @DimitrisPapail]

https://t.co/8p3sN1BMhS

[charlie @inthisframe]

@ThosVarley Reminds me of a blog post from a couple years ago on a similar issue. Probably a lot of “correct in spirit” work out there https://t.co/ECv84Ou08B

[Grigory Sapunov @che_shr_cat]

1/ Breakthrough science usually happens at the intersection of distant disciplines. But ask an LLM to generate a new idea, and it spits out the most heavily trafficked, predictable concepts. Here is how we can build an AI that thinks like an alien. 🧵

[charlie @inthisframe]

@maxbittker are the increments 99, 200M (in .1xp increments), and 2.14B? how does the model get told what its reward is?

[charlie @inthisframe]

@k7agar https://t.co/A9gLssxzTa

[charlie @inthisframe]

@stochasticchasm looks like an older shade of something from howlin https://t.co/QGCzRfYWrj

[Sierra Bedwell 🧬 @bedwellbacteria]

Super thrilled to announce that the first chapter of my PhD is now on BioRxiv!
https://t.co/Sp2ZcAvsEA

If you are interested in microbes, ecology, and evolution, please give it a read!

Long-ish 🧵:

[Goodfire @GoodfireAI]

LLMs memorize a lot of training data, but memorization is poorly understood.

Where does it live inside models? How is it stored? How much is it involved in different tasks?

@jack_merullo_ & @srihita_raju's new paper examines all of these questions using loss curvature! (1/7)

[Matthew Finlayson @mattf1n]

We discovered that language models leave a natural "signature" on their API outputs that's extremely hard to fake. Here's how it works 🔍

📄 https://t.co/Yc7mnhZS96 1/

[charlie @inthisframe]

@Tim38463182 @konstmish I see it's not in the paper, but have you done any experiments with the combo of cautious optimizers and cautious weight decay?

[computer man @gloomfather]

Visualized data is honestly so childish. it's basically the graphic novels of data. grow up lmao