Frances Ding

PhD student in EECS, UC Berkeley. ML fairness and interpretability. ML for protein design.

[Frances Ding @FrancesDing]

Our framework lets you:
✅ Benchmark any protein sequence model with just a few lines of code
✅ Get interpretable Elo ratings showing which species your model favors
✅ Compare against baselines in our paper
✅ Add your model results to expand the benchmark

Happy benchmarking!

[Frances Ding @FrancesDing]

Really excited by the response to our protein model species bias paper! Based on researcher interest and requests, we're releasing our benchmarking framework to make it easy to evaluate bias on any protein model of interest --> https://t.co/ePyhwjdluo

[Frances Ding @FrancesDing]

Protein language models (pLMs) can give protein sequences likelihood scores, which are commonly used as a proxy for fitness in protein engineering. But what do likelihoods encode?

In a new paper (w/ @JacobSteinhardt) we find that pLM likelihoods have a strong species bias!

1/

[Frances Ding @FrancesDing]

@miangoar @JacobSteinhardt Hug et al. is super interesting! Unfortunately their tree did not include all the species we studied, so that's why we created our own to visualize our dataset in particular.

[Frances Ding @FrancesDing]

Protein language models (pLMs) can give protein sequences likelihood scores, which are commonly used as a proxy for fitness in protein engineering. But what do likelihoods encode?

In a new paper (w/ @JacobSteinhardt) we find that pLM likelihoods have a strong species bias!

1/

[Frances Ding @FrancesDing]

@miangoar @JacobSteinhardt Thanks! To create our phylogenetic tree we used https://t.co/MCLtM4TMoS to get estimates of the time to last common ancestor between each pair of species we studied. Then we used hierarchical clustering to turn those estimates into a full tree.

[Frances Ding @FrancesDing]

@Juli_Bla Thank you! And thanks for the pointer, that's exciting work!

[Frances Ding @FrancesDing]

@biorxiv_bioinfo You can find a tweetorial overview of the paper here!: https://t.co/XxkOwdsC4r

[Frances Ding @FrancesDing]

Protein language models (pLMs) can give protein sequences likelihood scores, which are commonly used as a proxy for fitness in protein engineering. But what do likelihoods encode?

In a new paper (w/ @JacobSteinhardt) we find that pLM likelihoods have a strong species bias!

1/

[Frances Ding @FrancesDing]

Thanks for reading! For more details, check out the full paper here:

https://t.co/LXTl8IDW0S

14/14

[Frances Ding @FrancesDing]

More broadly, how should we structure and curate databases of biological data, which are not only repositories of knowledge, but now serve to define distributions over data?

13/