DongyuXue

Research Scientist at ByteDance Research.

[DongyuXue @dngyxu1]

How to steer generative antibody design models to design antibodies satisfying user-desired preferences, such as rationality and functionality🤔?
We are thrilled to introduce AbDPO, a general framework for antibody design via energy-based preference optimization.
The preprint, "Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization", is out here: https://t.co/W9nIUpfwuG .
Joint work w/ @Xiangxin_Zhou , @RobbenChen68 , @zaixiang_zheng , Liang Wang and @QuanquanGu .
Key takes:
1⃣ We revisit the antibody sequence-structure co-design from the perspective of preference optimization;
2⃣ We propose a residual-level energy-based DPO tailored for antibody design. There is a broad spectrum of choices of energy functions and we used Rosetta energy for verification;
3⃣ We decompose the energy into fine-grained terms and introduce conflict mitigation techniques to balance these terms;
4⃣ AbDPO successfully generates antibodies with comparable energy performance to natural antibodies.

[DongyuXue @dngyxu1]

congratulations!!!

[Yilai Li @li_yilai]

CryoSTAR is officially published in @naturemethods! Huge thanks to our reviewers for their valuable feedback. In this final version, we show that an AlphaFold-predicted atomic model may serve as the reference, showcasing cryoSTAR’s flexibility. https://t.co/Aox8wq78kL

[Yilai Li @li_yilai]

CryoSTAR is officially published in @naturemethods! Huge thanks to our reviewers for their valuable feedback. In this final version, we show that an AlphaFold-predicted atomic model may serve as the reference, showcasing cryoSTAR’s flexibility. https://t.co/Aox8wq78kL

[Yi Zhou @dugu9sword]

1/ 🔍 Wonder about the answer?
cryo-EM ✖️ Foundation Model 🟰 ❓
cryo-EM ✖️ Flow Matching 🟰 ❓
cryo-EM ✖️ Diffusion Transformer 🟰 ❓

Excited to introduce our new work--cryoFM, the first cryo-EM foundation model for protein densities with flow matching, which generalizes to four tasks without finetuning.

📃 Paper: https://t.co/YMyP2A50hu
🏠️ Project: https://t.co/O7OCzzduJF

Joint work with Yilai Li @li_yilai , Jing Yuan @eugenejyuan , Quanquan Gu @QuanquanGu

[Yuning Shen @YuningShen1]

🚀 Our latest work on ProteinBench is a living benchmark for protein foundation models! 🧬🔬
The medal for #NobelPrize chemistry went to protein design and structure prediction, and ProteinBench is an ongoing "Olympics" in these two domains.

It now includes 20+ models and 9 key challenges, from designing functional proteins to generating protein dynamic structures. Checkout it for more details:

🤗Leaderboard: https://t.co/iCDbAmwFNY
📄 Paper: https://t.co/bJSMSGJfYp
🌐 Website: https://t.co/vV3EiJocMX

We welcome community feedback and participation to help this benchmark continue to grow!

@QuanquanGu @zaixiang_zheng @dngyxu1 @YuningShen1 @leowang_1

[Biology+AI Daily @BiologyAIDaily]

PROTEINBENCH: A Holistic Evaluation of Protein Foundation Models

- PROTEINBENCH introduces the first comprehensive evaluation framework for protein foundation models, assessing their performance across a wide range of tasks like protein structure prediction, design, and conformational dynamics.

- The framework offers a taxonomic classification of tasks, helping researchers benchmark models for generative tasks such as protein design (including inverse folding, structure design, and sequence co-design) and antibody design.

- PROTEINBENCH evaluates models based on four critical dimensions: quality, novelty, diversity, and robustness, providing a more holistic view of model performance and capabilities across the protein space.

- Key insights reveal that while models like AlphaFold3 and the ESM series (e.g., ESM-IF1 and ESMFold) have achieved breakthroughs in structure prediction, the field still faces challenges in achieving consistent performance across all tasks, particularly in de novo design and multi-state conformation prediction.

- A public leaderboard and open-source code are available to promote transparency and foster collaboration, making PROTEINBENCH a living benchmark for the community to drive advancements in protein foundation models.
@QuanquanGu
💻Code: https://t.co/M5uCVfcLos
📜Paper: https://t.co/K5pOeuOuzW

[DongyuXue @dngyxu1]

@jianfcpku Yes! You are right🙋, thank you very much for pointing out this typo, and I appreciate your carefully reading our article❤️. If you have any other questions, please feel free to come to me for discussion~

[DongyuXue @dngyxu1]

How to steer generative antibody design models to design antibodies satisfying user-desired preferences, such as rationality and functionality🤔?
We are thrilled to introduce AbDPO, a general framework for antibody design via energy-based preference optimization.
The preprint, "Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization", is out here: https://t.co/W9nIUpfwuG .
Joint work w/ @Xiangxin_Zhou , @RobbenChen68 , @zaixiang_zheng , Liang Wang and @QuanquanGu .
Key takes:
1⃣ We revisit the antibody sequence-structure co-design from the perspective of preference optimization;
2⃣ We propose a residual-level energy-based DPO tailored for antibody design. There is a broad spectrum of choices of energy functions and we used Rosetta energy for verification;
3⃣ We decompose the energy into fine-grained terms and introduce conflict mitigation techniques to balance these terms;
4⃣ AbDPO successfully generates antibodies with comparable energy performance to natural antibodies.

[DongyuXue @dngyxu1]

5/ [Experiments Result]. Experiments on a variety of antigens demonstrate that AbDPO can effectively optimize multiple energies simultaneously and achieve performance comparable to natural antibodies. AbDPO achieves the SOTA performance of the average performance on each antibody-antigen complex in the RAbD dataset. Specifically, in the cases of 5mes and 1iqd, AbDPO achieved better energy performance than natural antibodies.

[DongyuXue @dngyxu1]

1/ [Challenge]. Data volume is a dilemma that antibody design tasks have to face.
~2000 non-repetitive antibody samples with structure are insufficient to support the model in the learning of maximizing the likelihood of natural antibodies and capturing the interactions between the antibody and the target antigen.
We found that this severe lack of data leads different antibody design methods to tend to generate antibodies exhibiting a fixed pattern (💡like "ARG+(Y/G)*n+FDY") in the complementary determining region (CDR), regardless of the specific antigens, and generating numerous structural clashes to the antigen.

[DongyuXue @dngyxu1]

4/ [Energy Decomposition and Conflict Mitigation]. We use two types of energy: the total energy that reflects the rationality of antibodies and the binding energy that indicates the functionality of antibodies. The binding energy is further decomposed into attraction energy and repulsion energy as the value of repulsion is usually several orders of magnitude larger than attraction.
Attraction and repulsion are mutually exclusive sometimes. Increased spatial proximity between the antibody and the antigen results in decreased attraction energy and increased repulsion energy (left figure), and vice versa (right figure). To resolve the conflict between these two energies and make AbDPO compatible with any combination of energy/properties, we apply "gradient surgery" to AbDPO to mitigate the tension between the above optimization objectives.

[Lihao Wang @leowang_1]

1/ Proteins exhibit a dynamic nature. We stand by the belief that steering with physical knowledge is vital in real-world dynamic structure prediction, and we're delighted to introduce our force-guided diffusion model for generating protein conformations. https://t.co/f4F1E9EDwq

[Zaixiang Zheng @zaixiang_zheng]

The recent surge of generative AI has been fueled by the powers of #LLMs and #Diffusion models.
🙋Wonder the answer of (LLM + Diffusion) x Protein = ?

Introducing DPLM (diffusion protein language model), a versatile protein language model that demonstrates strong generative and predictive capabilities for protein sequences.

Our preprint, "Diffusion Language Models Are Versatile Protein Learners", is out here:
https://t.co/ykGAPdIP3U

This is a joint work with our amazing team, Xinyou (@Xinyou_NJU), Fei (@FeiYE00844289), Dongyu (@dngyxu1), Shujian and Quanquan (@QuanquanGu).

DPLM is grounded in discrete diffusion framework and pre-trained from evolutionary-scale protein sequences.
As a result, DPLM blends the scalable expressiveness of language models and the strong generative power of diffusion models, taking the best of both worlds.

Some interesting findings we'd like to highlight:
📌 DPLM exhibits the ability to generate structurally plausible, novel and diverse protein sequences for unconditional generation.

📌 DPLM can serve as a superior protein representation learner, which can be fine-tuned for various predictive tasks, comparing favorably to ESM2.

📌 DPLM can be further tailored for various needs, demonstrating its prowess of conditional generation:
1️⃣ conditioning on partial peptide sequences, e.g., generating scaffolds for functional motifs with high success rate;
2️⃣ incorporating other modalities as cross-modal conditioners, e.g., structure-conditioned generation for inverse folding; and
3️⃣ steering sequence generation towards desired preferences, e.g., satisfying specified secondary structures, through a plug-and-play classifier guidance.

Finally, DPLM joins the ranks alongside ESM, EvoDiff, Evo and others, in harnessing generative modeling to decipher the Evolution of Life!!

1/n

[Quanquan Gu @QuanquanGu]

Give someone a fish, and you feed them for a day; teach someone to fish, and you feed them for a lifetime.
Elevating from Weak to Strong with Self-Play Fine-Tuning (SPIN) for All LLMs. Empower, Evolve, SPIN!

https://t.co/6xjvCxkKj9

[Quanquan Gu @QuanquanGu]

Kudos to @zaixiang93 and the incredible team for their great work in fixed backbone protein sequence design! 🧬Don't miss his talk on this work at the upcoming Machine Learning for Protein Engineering seminar @ml4proteins!

[Zaixiang Zheng @zaixiang_zheng]

1/ Wonder how language models can design proteins? Excited to share our recent work, Structure-informed Language Models Are Protein Designers, on harnessing large-scale protein language models (pLMs) for protein sequence design! This work has been published at #ICML 2023 with oral presentation.

In this study, we introduce LM-Design, a generic approach that transforms pLMs to protein design models via structural surgery. LM-Design generates favorable protein sequences for desired structures, offering a model-agnostic, modularizable, and parameter- as well as data-efficient approach.

This is a joint work with our amazing intern Yifan Deng (@Dengyifan1012), together with Dongyu Xue (@dngyxu1) , Yi Zhou (@dugu9sword), Fei Ye (@FeiYE00844289) and @QuanquanGu.

Paper: https://t.co/tipzK6oJWY
Code: https://t.co/xA8E2HzUzO

Also, we are working on the next-generation LM-based sequence design toolkit, LM-Design 2.0. Stay tuned!

[Quanquan Gu @QuanquanGu]

📢 Excited to share our latest research on improving human-AI communication! 🤖💬 We introduce 'Rephrase and Respond' (RaR), a simple yet effective method that enhances LLMs’ understanding of human questions. Check out how RaR improves #GPT4 performance by resolving ambiguities & can be integrated with Chain-of-Thought (#CoT) for more robust AI responses.

🌟This work is led by @Yihe__Deng and an exceptional team of students @WeitongZhang @_zxchen_

Paper: https://t.co/NXcE4WNvSo
Project: https://t.co/QOzXpQ5j4n
Code: https://t.co/TQN1XYcW4X
HuggingFace: https://t.co/M3ClYX8Ang

Key Insights:
👉 Human input is key to LLM response quality. Crafting clear, detailed questions is crucial as our different thought frames may lead to AI misunderstandings.
👉 To tackle the disparity between human and LLM thought frames, we introduce RaR, which prompts the LLM to rearticulate the given question, and respond.
👉 Our experiments demonstrate that RaR significantly improves the performance of various GPT models across a wide range of tasks.
👉 We introduce formal mathematical formulations for both CoT and RaR and show that RaR is different and complementary to CoT. Through empirical analysis, we illustrate the importance of question quality—it should be prioritized before enhancing the model’s reasoning capabilities! 🧵1/N

[Quanquan Gu @QuanquanGu]

Very excited to be at the helm of the AI for Science initiative at ByteDance Research. Our unwavering commitment to reshaping the scientific discovery landscape using AI is a journey to watch.

Today, we release CryoStar, a state-of-the-art open-source tool for Cryo-EM heterogeneous reconstruction. It's the epitome of innovation in merging AI with the world of structural biology.

Project: https://t.co/xDQHqyaqE4
Code: https://t.co/xI6aPRsb3s
Paper: https://t.co/kUyuXi3R0w

Join us on this remarkable journey of exploration, from AI breakthroughs to groundbreaking discoveries in science. Stay connected, as more models and open-sourced tools are on the horizon!