![amansplaining's tweet photo. 2 days ago Evolutionary Scale launched ESM3, a protein language model that can generate novel proteins and a lot more
The numbers:
• 98 billion parameters
• 2.78 billion natural protein sequences in the training set
• 771 billion unique tokens of training data
What's the hype about?
[1] It speaks fluent protein
ESM3 doesn't just understand protein sequences - it grasps structure and function too. This multimodal (sequence, structure, and function) approach is a first.
[2] Architecture custom-built for protein data
It's a masked language model specifically designed to handle the complexity of protein data:
• Multi-track input/output: Processes protein sequence, structure, and function as separate token tracks.
• Geometric attention: The first transformer block includes a special geometric attention layer to handle 3D atomic coordinates.
• Masked prediction: During training, random masks are applied across all tracks, and the model learns to predict the masked tokens.
[3] 500 million years of evolution in one model
ESM3 generated a novel green fluorescent protein (GFP) that's so different from known proteins, that it's comparable to what we'd expect to see after 500 million years of natural evolution.
[4] Data quality is still king
The team didn't just rely on natural proteins. They augmented their dataset with synthetic examples to give ESM3 an edge.
[5] Chain of thought isn't just for language anymore
They used chain-of-thought prompting to guide ESM3 in protein design. NLP techniques are proving their worth even in hard sciences.
The team is also releasing ESM3-open, a 1.4B parameter model, for academic research.
ESM3 is the best example of the ideas behind language models being applied beyond language and code.
Kudos to @alexrives and team! 🙌](https://pbs.twimg.com/media/GRFyRlFXMAAZlhp.png)
![amansplaining's tweet photo. 2 days ago Evolutionary Scale launched ESM3, a protein language model that can generate novel proteins and a lot more
The numbers:
• 98 billion parameters
• 2.78 billion natural protein sequences in the training set
• 771 billion unique tokens of training data
What's the hype about?
[1] It speaks fluent protein
ESM3 doesn't just understand protein sequences - it grasps structure and function too. This multimodal (sequence, structure, and function) approach is a first.
[2] Architecture custom-built for protein data
It's a masked language model specifically designed to handle the complexity of protein data:
• Multi-track input/output: Processes protein sequence, structure, and function as separate token tracks.
• Geometric attention: The first transformer block includes a special geometric attention layer to handle 3D atomic coordinates.
• Masked prediction: During training, random masks are applied across all tracks, and the model learns to predict the masked tokens.
[3] 500 million years of evolution in one model
ESM3 generated a novel green fluorescent protein (GFP) that's so different from known proteins, that it's comparable to what we'd expect to see after 500 million years of natural evolution.
[4] Data quality is still king
The team didn't just rely on natural proteins. They augmented their dataset with synthetic examples to give ESM3 an edge.
[5] Chain of thought isn't just for language anymore
They used chain-of-thought prompting to guide ESM3 in protein design. NLP techniques are proving their worth even in hard sciences.
The team is also releasing ESM3-open, a 1.4B parameter model, for academic research.
ESM3 is the best example of the ideas behind language models being applied beyond language and code.
Kudos to @alexrives and team! 🙌](https://pbs.twimg.com/media/GRFyN0_WcAEE-uJ.png)
![amansplaining's tweet photo. 2 days ago Evolutionary Scale launched ESM3, a protein language model that can generate novel proteins and a lot more
The numbers:
• 98 billion parameters
• 2.78 billion natural protein sequences in the training set
• 771 billion unique tokens of training data
What's the hype about?
[1] It speaks fluent protein
ESM3 doesn't just understand protein sequences - it grasps structure and function too. This multimodal (sequence, structure, and function) approach is a first.
[2] Architecture custom-built for protein data
It's a masked language model specifically designed to handle the complexity of protein data:
• Multi-track input/output: Processes protein sequence, structure, and function as separate token tracks.
• Geometric attention: The first transformer block includes a special geometric attention layer to handle 3D atomic coordinates.
• Masked prediction: During training, random masks are applied across all tracks, and the model learns to predict the masked tokens.
[3] 500 million years of evolution in one model
ESM3 generated a novel green fluorescent protein (GFP) that's so different from known proteins, that it's comparable to what we'd expect to see after 500 million years of natural evolution.
[4] Data quality is still king
The team didn't just rely on natural proteins. They augmented their dataset with synthetic examples to give ESM3 an edge.
[5] Chain of thought isn't just for language anymore
They used chain-of-thought prompting to guide ESM3 in protein design. NLP techniques are proving their worth even in hard sciences.
The team is also releasing ESM3-open, a 1.4B parameter model, for academic research.
ESM3 is the best example of the ideas behind language models being applied beyond language and code.
Kudos to @alexrives and team! 🙌](https://pbs.twimg.com/media/GRFyHBYW4AApKF5.png)
Olivia
Online
Aman Dhesi
@amansplaining
@WeAreLegora
NYC
Joined March 2012
674 Following
1.1K Followers
1.2K Posts
Pinned Tweet
Today, I'm excited to announce that @WeAreLegora has acquired Cadastral.
Cadastral is an AI agent platform built specifically for commercial real estate, trusted by JLL, AvalonBay, Equity Residential, and Empire State Realty Trust. In just over a year, they've signed 50+ firms and grown revenues by 40% per month on average.
Why CRE? Because it may be the most legally intensive industry on earth. Acquisitions. Leases. Refinancings. Disputes. Every deal produces a wall of documents that demands precision. Legal teams here have never had AI built for them, until Cadastral.
Co-founders Abe Somani and Aman Dhesi and their NYC engineering team are joining Legora, and planting the flag for our first major US engineering hub. We're building toward 200+ people in New York and 300+ across North America by end of 2026.
This is our fourth acquisition this year. The thesis is consistent: Legora is an agentic operating system for legal work wherever that work happens. Law firms. In-house teams. And now the industries that create the most complex legal work in the world.
CRE is next. It won't be the last.
Welcome, Abe, Aman, and the Cadastral team.
Full story:
https://t.co/OIHu0DWPqR
@benscharfstein thank you!
Introducing Vesta
The AI for your closet
Vesta gives you daily outfits based on your wardrobe, your schedule, the weather, and your style.
The days of having “nothing to wear” are over.
(Re)Introducing Highlight: The Marketplace For Believers.
NFTs aren't just digital ticker symbols—they're vibrant expressions of human culture, creativity, and community.
Here's what's new on Highlight ↓
Who to follow
Aditya Agarwal 
@adityaag
General Partner @SouthPkCommons, Co-Founder @Bevel_Health | Ex: Early Eng @facebook, CTO @Dropbox, Board @Flipkart | Optimist, Builder, Dad
Ben Scharfstein
@benscharfstein
building @appliedcompute. started a few companies. previously product @scale_ai, @google, applied math @harvard
Daniel Mason
@dgmason
Building @anonplatform. 3x founder; angel investor (25+ companies)
@MichaelDean_0 @osventuresllc Congrats Michael! So happy for you
I get a twisted sort of pleasure from seeing TSLA shorts get burned. Go ahead, short the most important company of our generation 🤷♂️
17.6% of hedge funds are now shorting Tesla, $TSLA, the highest share in over a year.
This is up from 13.9% recorded three months ago, according to Hazeltree which tracks over 500 funds.
Last week, Tesla's short sellers lost $3.5 billion in just 2 days the company beat expectations for Q2 2024 vehicle deliveries.
Since the April 2024 low, the stock is up almost 80% and added ~$350 billion in market cap.
It is now up 1% year-to-date after initially falling 42% in the first 4 months.
Incredible turnaround.
2 days ago Evolutionary Scale launched ESM3, a protein language model that can generate novel proteins and a lot more
The numbers:
• 98 billion parameters
• 2.78 billion natural protein sequences in the training set
• 771 billion unique tokens of training data
What's the hype about?
[1] It speaks fluent protein
ESM3 doesn't just understand protein sequences - it grasps structure and function too. This multimodal (sequence, structure, and function) approach is a first.
[2] Architecture custom-built for protein data
It's a masked language model specifically designed to handle the complexity of protein data:
• Multi-track input/output: Processes protein sequence, structure, and function as separate token tracks.
• Geometric attention: The first transformer block includes a special geometric attention layer to handle 3D atomic coordinates.
• Masked prediction: During training, random masks are applied across all tracks, and the model learns to predict the masked tokens.
[3] 500 million years of evolution in one model
ESM3 generated a novel green fluorescent protein (GFP) that's so different from known proteins, that it's comparable to what we'd expect to see after 500 million years of natural evolution.
[4] Data quality is still king
The team didn't just rely on natural proteins. They augmented their dataset with synthetic examples to give ESM3 an edge.
[5] Chain of thought isn't just for language anymore
They used chain-of-thought prompting to guide ESM3 in protein design. NLP techniques are proving their worth even in hard sciences.
The team is also releasing ESM3-open, a 1.4B parameter model, for academic research.
ESM3 is the best example of the ideas behind language models being applied beyond language and code.
Kudos to @alexrives and team! 🙌
![amansplaining's tweet photo. 2 days ago Evolutionary Scale launched ESM3, a protein language model that can generate novel proteins and a lot more
The numbers:
• 98 billion parameters
• 2.78 billion natural protein sequences in the training set
• 771 billion unique tokens of training data
What's the hype about?
[1] It speaks fluent protein
ESM3 doesn't just understand protein sequences - it grasps structure and function too. This multimodal (sequence, structure, and function) approach is a first.
[2] Architecture custom-built for protein data
It's a masked language model specifically designed to handle the complexity of protein data:
• Multi-track input/output: Processes protein sequence, structure, and function as separate token tracks.
• Geometric attention: The first transformer block includes a special geometric attention layer to handle 3D atomic coordinates.
• Masked prediction: During training, random masks are applied across all tracks, and the model learns to predict the masked tokens.
[3] 500 million years of evolution in one model
ESM3 generated a novel green fluorescent protein (GFP) that's so different from known proteins, that it's comparable to what we'd expect to see after 500 million years of natural evolution.
[4] Data quality is still king
The team didn't just rely on natural proteins. They augmented their dataset with synthetic examples to give ESM3 an edge.
[5] Chain of thought isn't just for language anymore
They used chain-of-thought prompting to guide ESM3 in protein design. NLP techniques are proving their worth even in hard sciences.
The team is also releasing ESM3-open, a 1.4B parameter model, for academic research.
ESM3 is the best example of the ideas behind language models being applied beyond language and code.
Kudos to @alexrives and team! 🙌](https://pbs.twimg.com/media/GRFyUNvWgAEpzoQ.png)
I hope you've found this thread helpful.
Follow me @amansplaining for more threads like these.
Like/Repost the quote below if you can:
Imagine a world where factories aren't locked into producing a single part for decades
Where a production line can shift from car doors to airplane wings overnight
This isn't sci-fi. It's roboforming, and it's happening right now in LA.
read more 🧵
Imagine a world where factories aren't locked into producing a single part for decades
Where a production line can shift from car doors to airplane wings overnight
This isn't sci-fi. It's roboforming, and it's happening right now in LA.
read more 🧵
Shoutout to the @MachinaLabs_ team for doing something extraordinary.
Sources I used to write this thread:
https://t.co/MVH3sqvtuW
https://t.co/LFeorLxa9Y
https://t.co/Pq7lCKu40B
https://t.co/eAiPSGkMOM
Last Seen Users on Sotwe
Trends for you
Most Popular Users
Elon Musk
@elonmusk
240.2M followers
Barack Obama
@barackobama
119.3M followers
Donald J. Trump
@realdonaldtrump
111.6M followers
Cristiano Ronaldo
@cristiano
109.4M followers
Narendra Modi
@narendramodi
106.9M followers
Rihanna
@rihanna
97.4M followers
NASA
@nasa
92.1M followers
Justin Bieber
@justinbieber
90.7M followers
KATY PERRY
@katyperry
87.1M followers
Taylor Swift
@taylorswift13
80.9M followers
Lady Gaga
@ladygaga
72.4M followers
Kim Kardashian
@kimkardashian
69.5M followers
Virat Kohli
@imvkohli
69M followers
YouTube
@youtube
68.6M followers
Bill Gates
@billgates
63.5M followers
The Ellen Show
@theellenshow
62.5M followers
CNN
@cnn
61.9M followers
Neymar Jr
@neymarjr
61.6M followers
X
@x
60.9M followers
Selena Gomez
@selenagomez
60.2M followers