I love rethinking components of AI models, just like from MLPs to KANs. This time we propose an alternative of the cross entropy loss — the harmonic loss, inspired by Euclidean geometry! Harmonic loss is a stone killing three birds: data efficiency, grokking and interpretability!
One of the most interesting things is seeing developers struggle to come up with a cognitive model of what they are doing when setting up multi agent systems
The Scaling Paradox:
AI capabilities have improved remarkably quickly, fuelled by the explosive scale-up of resources to train the leading models. But the scaling laws that inspired this rush actually show very poor returns to scale. What’s going on?
1/
https://t.co/gm32S8SpDX
every time i remember we can do this i get a rush of excitement to be alive. literally just do dimensionality reduction w/ UMAP on neuronal spike data and you can recover topology from mental models of physical space
Decoding the Molecular Language of Proteins with Evola
1. Evola introduces an 80-billion parameter multimodal protein-language model to decode protein functions, leveraging protein sequences, structures, and user queries.
2. A key innovation is its unprecedented training dataset: 546 million AI-generated protein question-answer pairs with 150 billion word tokens, reflecting immense protein diversity.
3. Evola integrates advanced techniques like Direct Preference Optimization (DPO) for model refinement and Retrieval-Augmented Generation (RAG) for incorporating external knowledge, ensuring high-quality, nuanced responses.
4. The Instructional Response Space (IRS), a novel evaluation framework, showcases Evola’s expert-level performance in protein annotation tasks like enzyme classification and gene ontology prediction.
5. The model outperforms general-purpose LLMs, demonstrating a nearly twofold improvement in generating precise, protein-specific insights compared to GPT-4-like models.
6. With scaling capabilities, Evola demonstrates enhanced performance by leveraging larger datasets and model sizes, culminating in Evola-80B achieving superior generalization on unseen protein data.
7. Evola’s ability to interpret protein molecular mechanisms extends applications to drug discovery, functional genomics, and biomedical research, revolutionizing protein functional understanding.
@duguyuan@LTEnjoy@XibinBayesZhou@ChenchenHa42849@shiyu_jiang23
📜Paper: https://t.co/bkPyk4cL2A
#Proteomics #AI #ProteinLanguageModel #FunctionalGenomics #Biotechnology
A DNA language model based on multispecies alignment predicts the effects of genome-wide variants https://t.co/NFn1dyxBxf (read free: https://t.co/jWEs4Lvh3g) 🧬🖥️🧪 https://t.co/f2W6QiCslr
Are you teaching a chemistry class in the spring semester? Do you want to add computation to your course, but don't know how to do so?
Rowan makes it fast and easy to bring modern computations into problem sets, in-class activities, or virtual labs. (🧵)
📢Thrilled to introduce the #VirtualLab: a team of AI scientist agents (AI chemist, AI reviewer...). Virtual Lab is led by an AI professor w/ feedback from human scientist.
The Lab created new nanobodies that we experimentally validated to bind to recent #covid variants🚀🧵