Models cost millions to train and minutes to copy. Fine-tuning and distillation can erase where they came from.
Introducing the Fingerprinting Leaderboard, an open benchmark for model provenance — built so the proof lives in the weights, not the paperwork.
https://t.co/aHONvNd7o4
Models cost millions to train and minutes to copy. Fine-tuning and distillation can erase where they came from.
Introducing the Fingerprinting Leaderboard, an open benchmark for model provenance — built so the proof lives in the weights, not the paperwork.
https://t.co/aHONvNd7o4
Why are there three leaderboard scores?
Model provenance carries clues about the history, training data, development process, and later modifications. Fine-tuning and distillation obscure provenance to the point where it’s virtually impossible to track the origin of the model back to its creator.
And if you as an owner want to reliably prove attribution and claim ownership of your open weights, you should choose a fingerprinting scheme based on these three dimensions:
>> Retention: whether the fingerprint is detectable without sacrificing normal model quality.
In other words, how embedding fingerprints affects model performance compared to the unfingerprinted model's nominal performance.
>> Robustness: whether the fixed attack can remove the fingerprint, preserving output quality.
In other words, how resistant the fingerprinted model is to malicious attempts to obscure its origin.
>> Verification: whether model owner/creator can reliably read the fingerprint back from a suspected copy and verify his IP property.
In other words, how reliable model attribution would be under scrutiny/inspection.
opening a model shouldn’t mean giving up the ability to prove it’s yours.
that’s the problem we’re tackling at @sentient_found.
this leaderboard tests how well model fingerprints survive attacks and how much model quality they preserve.
This is the missing piece for real open-source AI.
Training costs millions; copying takes minutes. Embedding provenance directly into model weights, rather than relying on paperwork, means models can remain open without losing their origin or credit.
The historical parallels are spot on, and an open leaderboard is exactly what this research needs to move forward.
A big step for open-source AI.
Kudos to the @SentientAGI for mmaking this possible.
as AI models become easier to copy, fine-tune, and distill, proving where they came from matters more than ever.
renaissance painters left hidden artist marks on their masterpiece and now, open source ai creators and builders should have a way to mark the intelligence they build & prove where it came from. 🤗
1/ In collaboration with the University of Washington, Princeton University, and the University of Illinois at Urbana-Champaign, we evaluated 14 fingerprinting schemes created by Harvard, UCLA, Oxford, Columbia, Microsoft, and Meta on three core dimensions:
- affect on performance;
- resistance to malicious intent;
- and reliability of attribution.
Foster responsible intellectual property handling with our Fingerprinting Leaderboard 👇 🧵.
• Lexicographers planted fake words in dictionaries.
• Cartographers drew fake streets into maps.
• Artists hid their signatures inside the paint.
• AI builders embed fingerprints into their models.
History doesn't repeat itself, but it does rhyme.
EvoSkill made it into Franklin Templeton’s latest report, and @PreyWebthree was ready with his take.
For him, the question is no longer which model is better, but what you can build on top of it ↓
https://t.co/CtOFQiEl9s
The real power of open models may not lie in the model alone.
How well a model performs matters, but so does what is built around it: skills, harnesses, and mechanisms that allow it to learn and improve.
Franklin Templeton’s report highlights Sentient’s EvoSkill as an interesting example of this shift.
The idea is simple but powerful: agents can learn from their past tasks and turn that experience into skills they can reuse on future tasks.
This suggests that higher performance can be extracted from lower-cost open models through the systems built around them.
So the competition in open AI may not simply be about “which model is better?”
It may increasingly be about what can be built on top of the model.
@SentientAGI@sentient_found@SentientEco