The 8B model currently training on Agora is 350B tokens in and continuing to converge. The top level metrics and evals look almost exactly like a centralised run. But;
- 133 external contributors total bringing 4090's, 5090's, L40S/RTX 6000 and RTX 6000 Pros. These are cards that people actually own - there are no H100, B200's etc.
- The max number of nodes the system can support (104) was filled almost immediately. The authorization layer is receiving approximately 100 requests/minute to join.
- The total tokens/per second processed moves directly with amount of compute in the swarm, with Agora constantly optimising to make most efficient use of what hardware is present.
- MFU is approximately 20%, TPS is 170k tok/s. There are near constant communication failures which Agora is completely absorbing without slowdown.
- The system is effectively on auto-pilot, requiring very little intervention from us. Bad nodes are purged immediately before training is affected and new nodes take their place.
Nous Research announces the pre-training of a 15B parameter language model over the internet, using Nous DisTrO and heterogeneous hardware contributed by our partners at @Oracle, @LambdaAPI, @NorthernDataGrp, @CrusoeCloud, and the Andromeda Cluster.
This run presents a loss curve and convergence rate that meets or exceeds centralized training.
Our paper and code on DeMo, the foundational research that led to Nous DisTrO, is now available (linked below).
I felt bored and vibe-coded a neobank in a week. The best part? You don’t even realize it runs entirely on crypto rails under the hood.
Main functionalities:
- On/off-ramp fiat to stables via BridgeXYZ
- Send, receive, and request stablecoin payments on Base
- Deploy funds to selected blue-chip DeFi protocols
- Get a free Gnosis Pay crypto debit card
All of this is powered by account abstraction and non-custodial wallets.
Users receive a Privy wallet upon registration and never have to worry about signing transactions or approvals.
Transactions are sponsored and batched via EIP-7702, so no separate token approvals are required either.
In a nutshell: one-click access to stablecoin transactions sent directly to users via email, plus seamless entry into DeFi.
Tech stack:
- BridgeXYZ – on/off-ramp fiat to stables
- Privy – non-custodial wallets
- EIP-7702 – transaction batching and sponsoring
- iOS app built with React Native
- LIFI API – swapping between USDC and EURC
- Resend – email notifications
- DeFi protocols: Aave, Morpho, Compound, and YO
- Gnosis Pay API – virtual crypto debit cards
- NodeJs Backend
Inspired by @alliance's request for startups.
If you want to check it out, let me know so I can add you to TestFlight.
some people make the mistake of thinking they're cheating because they're crazed ideologues
there is an element of truth to this but there is an even better explanation for a group of people feeling the existential need to cheat:
funding and criminality
money and self preservation are much stronger motivators than ideology will ever be
these are deeply corrupt people who can't stop
they have to double down
pratt, like trump, are important political figures on a variety of dimensions but the most important might be their revelatory function
they highlight the need to fight to win especially when the structural forces against you are immense
really playing to win, not just larping, tests and reveals the structure unlike playing nice with it or rationally conceding to an inevitable conclusion
the goal is always to win
pratt losing is not the outcome we want but by him putting the structural forces to the test he has revealed and tested the structure far more than it is comfortable with
by doing so has created an opening for a process of delegitimization and restructuring that opens up the possibility of a win in the future where there was virtually none before
pratt has moved the ball forward against impossible odds; a strong consolation prize that can create the necessary momentum to eventually winning the prize
Somebody else needs to help me do this with charts and graphs but I’ll try with words first.
- In a normal election, you have 2 candidates, R & L. People cast votes. Proportionally, the votes of each candidate rise and fall in little chunks but take an overall steady line due to the force of averages. One of the lines eventually tops out, and that candidate wins on election night.
- In a mail in election, you have 2 candidates R & L. In this instance, some mail ins (if allowed) may come in after Election Day. But still, due to the force of averages, their lines with have little ups and downs but maintain a steady upward trajectory both during and after the election until one wins.
Now it has become the norm that after the election in mail in states, Dem candidates tend to go from tied or losing to drastically ahead AFTER the election, against the force of averages. How?
There are two legal, non fraudulent explanations:
1) that “Dems vote late by mail.” Dubious, considering that it used to be that Rs were known as absentee voters, but technically possible that for some never explained reason, Ds vote later than Rs by mail and that explains the statistical anomaly of why Dems lines shoot up after Election Day.
2) ballot harvesting. The far more likely explanation. Legal in California, this is when huge amounts of ballots are collected by political activists (unions, NGOs etc) and submitted on behalf of large populations of people in churches, workplaces etc. Both sides do it, but it is technically possible that Dems are much better at it. This is how insiders explain the “red mirage” statistical anomalies in 2020 and that have become so common since. Also explains why the prediction markets clinched Raman as winning while she was still down.
However, there is something different happening in Pratt v Bass election. A third candidate, a second, much less popular and less well known Dem is in the race, Nithya Raman. And this time, the late mail in ballots—mailed on Election Day or just before which is why they are “late mail in”—didn’t go to “the dems” they went specifically to Nithya Raman. Pratt stayed flat. Bass stayed flat. Nithya went way up after the initial count was reported. Unprecedented.
Oddly, the mail in mailed any time before Election Day did not favor nithya at all. Only the late mail ins counted after initial numbers were reported massively favored her. So we’re meant to believe that Nithya voters specifically voted not just by mail by late by mail, while the other Dem and Pratt voters did not. Bass voters stopped voting late mail in entirely, as they had last election against caruso, and Nithya voters started?
Could it be “Dems vote late by mail?” No, because then both Nithya and Bass, both Dems, would have risen at roughly the same rate, only without Pratt. But nithya massively over performed just enough to oust pratt, which would not have been possible had she and bass split the votes.
Could it be ballot harvesting? Also no. You can harvest a church or workplace and predict roughly that your harvested votes will favor Dems, but how could you feasibly predict which Dem out of two? Nithya’s support, always squarely in third place during the polls, is mostly educated white women, definitely not the type to vote late or be harvested. But that’s actually besides the point. How could a vote harvester, who is supposedly delivering filled out and sealed ballots from a constituency the harvester believes will lean left, know which left wing candidate this or that batch favors? And moreover, why would they then send only those batches after the election, while submitting all the other mail in ballots for bass before the election?
Occams Razor points to the much simpler explanation. Whoever is harvesting the votes isn’t just harvesting them, they’re somehow controlling the votes themselves, whether by filling them out or throwing away competing votes they don’t like. This is the definition of voter fraud.