Unlike many investors in crypto, I did not pivot to AI in the last few years. However, since 2020, I built some of the deepest understanding in this industry on the intersection of AI and decentralized networks (crypto, web3).
From the start, it was very clear that AI models are a centralizing force and the biggest target for government control. That point became market fact last night, with @AnthropicAI’s export control compliance.
As an investor in decentralized AI, I know that d-networks are a counterbalance to this state of affairs. In particular, the starting point of sovereign, open, public, decentralized AI is the seemingly insurmountable compute problem.
How are people supposed to source more industrial compute for frontier training than these huge trillion dollar companies? The answer is simple: there is enough commodity GPU compute in the world to compete on the frontier, but to make use of it we need new algorithms for training.
That’s what a few companies like @gensynai@PrimeIntellect@bageldotcom@Pluralis@NousResearch@MacrocosmosAI@covenant_ai set out to research, while everyone on the planet told them it was impossible.
The result is that it is not only possible, but it can be cheaper and nearly as efficient as the alternative process.
The second major problem is economic sustainability. Open source models are great, however, they are not economically viable as they don’t have a business model. So far in decentralized AI, only @Pluralis has an answer — by breaking up the weights of the model among participants, we create a business model for tokenized AI models.
This is the moment of truth — will AI become fully centralized and fall under censorship and unilateral government control? Or will the AI world realize the importance of public AI on open decentralized networks?
Today we're releasing Agora: the first ever pretraining stack that allows non-collocated consumer GPUs to be competitive with centralized clusters
Agora is 15x faster than Megatron-LM in this setting and is only 1.5x less efficient in terms of tokens per unit compute than TorchTitan on H100s, despite running on devices that have no NVLink or InfiniBand support.
@AshleyLLouise@garrytan Sure but what's also net negative for communities is being squashed by China because they zoom ahead of us, because we put a moratorium on data centers and fall behind on compute
It depends on the project but I broadly disagree here
I think any non-speculative tokenholders are also expecting to accrue future profits, and also have trust in management. Obviously that trust on both fronts is often misplaced, hence the dark view - but while buybacks prove some goodwill, if you're buying most tokens purely for buybacks and not future turbogrowth, I don't think that's a great idea. And to Guy's point, that turbogrowth could be much more easily achieved if they redirected buybacks into growth. At EOD these are startups and don't defy the laws of physics or follow dramatically different rules to those in web2, despite token-related differences
Re: tokens --> faster growth, I don't think this is true overly often anymore, look over at the AI party and at the struggles many DeFi companies are going through (and often flattening traction post TGE) despite token incentives
The sentiment though that we need more mechanisms for trust in token projects' good faith in both growing and accruing value to the token is one I strongly share, though, and will go to converging these two views
@0xJeff This may be true today but would be surprised if it holds. M2M is way too big of an opportunity for Stripe to lay down and let someone else take
agreed though that Stripe's advantage is more so in M2V, so they def start there
@KyleSamani It morphs into a game of chance when you systematically ban all the sharps, which is the strongest argument for the difference between sports PMs and sportsbetting I've seen
There's a lot of novel questions about what will accrue value, have moats, or be attractive if the target you're selling to is an agent vs. a human, which I (maybe weirdly) find fun to debate. There's also a lot of paths it can go, vs. other fields where the winning vectors are more straightforward
There's also a credible chance (nowhere near a given) that agents prefer crypto rails en masse, which would make this one of a select number of areas crypto could break out into the mainstream in the next couple of years - main reason I'm in the space is to find those. Though that's more idealism vs intellectual
Agents generally are also just a fast moving and exciting space. I found using Claude Code for the first time more mindblowing than anything else I'd seen in tech, incl. seeing ChatGPT come out. Forming an essential part of the stack, even if it's invisible to everyone but the agent, is cool
Not to mention the bottom now essentially has a Buzzfeed article. Cancelled sub
"Want to know the top 3 things people get wrong when they're in your exact position?"
There's a lot of really exciting developments happening in decentralized AI training this year. Here's my take on why decentralized training is moving from "impossible" to "investable". 🧵👇