I'm very excited to see more people in the open source ecosystem raise their ambition for what they can do!
Many lament the loss of old architectures and techniques but the excuses they have for not just training them themselves are diminishing more and more by the day
Training entirely new models has been the domain of academics & well-funded startups
But thanks to LLMs & the knowledge proliferation, people w/ a deep understanding of how models are actually used are increasingly equipped to train them
Great early example of this from @sin_ceriously & @Machinedelusion:
Zero-shot cloning has never been enough for studio work.
So we made our own model: Robin
A fully opensource 48khz English and Japanese voice model. (links below)
An Autoregressive architecture from scratch and designed with the OS community in mind from the start.
I’m a fan of Hozier & Jessie Buckley, but this is an great example of what aspiring to be part of the mainstream Irish cultural elite does to art
Every jagged edge, not just lyrically but musically & vocally, is smoothed away until what remains isn’t far from mediocre AI music
Fairytale of New York with Jessie Buckley is out now!
When I first heard that this project was being put together and that of all songs featured I might end up tackling 'Fairytale of New York', I'll admit to feeling somewhat daunted by the prospect. It is such a beloved song in Ireland it can seem as though some part of it belongs to the island as a whole.
@ShaneMacGowan is one of those rare artists that was even in his lifetime held in something of a mythologised light. His work is so intimately woven into the tapestry of our country’s musical canon and so definitive of a mode of musical being that it is truly foundational for so much that has followed. It was an honour to be asked to contribute to the record, and indeed to be trusted with this song.
Reaching out to Jessie Buckley out of the blue with the impossible ask of joining me on this recording is likely the most brass my neck has ever exhibited. Jessie was utterly skybound at the time, moving at a right angle to the earth and about to break the sound barrier, all while caring for a newborn baby and carrying every ounce of it all with a lightness of being that would put any tour-weary musician to shame.
In the busy aftermath of winning the Oscar for Best Actress for her incredible performance in Hamnet, Jessie still somehow managed to make time for this duet, bringing every ounce of her grace, energy and talent to the studio in London. To sing with her on this track, and to do so for such a good cause in aid of Dublin’s Simon Community has been an immense joy. - Hozier
To step into the very studio that Shane, Kirsty and the Pogues recorded “Fairytale of New York” and to sing this eternal song of all our hearts, full of hope and humanity was pure joy. When Hozier, one of Ireland’s greatest artists and souls asked me to be part of it, I felt like the luckiest woman in the world. What a pleasure! What a song! What a gift to get to sing this story! - Jessie Buckley
Honouring Shane MacGowan’s legacy, artist proceeds will be donated to Dublin Simon Community to provide lifesaving housing and healthcare for those experiencing homelessness.
https://t.co/Gug1GmptE5
@martinvars You seem to be falling for WSJ’s dishonest framing
It was very common knowledge that there are many datacenters in Malaysia, Singapore, etc. used by Chinese companies - and it was allowed by the US govt. and not illegal or even punished by trade policy
@krishnanrohit@azeem Number of people who are lying by selective presentation of facts w/o appropriate context: 📈
Number of media institutions that seem committed to doing this in lock-step: 📈
@AnMailleach I actually am not against people covering it but it feels like they just don’t understand it at all
Like when you remove everything that makes something what it is w/o any kind of opinion, idea, or reinterpretation, what are you even doing?
He didn’t include Deepseek’s latest models in the tweet/graphic because there wasn’t evidence that they were trained NVIDIA GPUs
Instead, he chose to include Deepseek V3 despite it being public information that that were trained on H800s
Profoundly dishonest framing
“But if you scroll down to the 8th paragraph of the 4th section, you’ll find…”
Give me a break
I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek��s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at https://t.co/YOtzND0Jqk.
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
@teortaxesTex@boneGPT@willdepue Profoundly dishonest to also not mention the models that didn’t align with his thesis in the tweet/communication
Absolute hack
@patio11 The framing is profoundly dishonest, he omits models that don’t fit the narrative from the communication and tweet, burying this deep in it
If it’s journalism, it’s hack journalism
@willdepue Have you heard of “lying by omission”
Profoundly dishonest to omit from the headline - especially when you include a report of the providers and their models but omit the ones that don’t fit your narrative
My vote for new @nvidia CEO is going to have to be @ClementDelangue . To me, he is the obvious choice given the recent Hugging Face acquisition. Plus, everyone in the AI/ML field seems to universally respect him.