I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
When I said hunch could learn in real-time I wasn't kidding. It's like teaching a kid!
Behind the scenes, @Cloudflare’s Clef is helping refine the local model so it learns in real time, while @googlegemma’s EmbeddingGemma2 runs live on the CPU. 🧵
I strongly suspect that memory, produced by electrical waves in our brains, is something like this. We may only be recording memories imaginatively, but in a way that closely resembles the visual original.
tomorrow is the last ever Pythagorean theorem day that we will ever get to experience. There’s only ever been 12 of them ever in our lifetime. And they will never be one ever again. How fun!🙂
Last month, we demonstrated high-rate alchemy: turning mercury into real, stable gold, beating CERN’s record by a factor of 10.
Last week, we enriched isotopes in the world’s most powerful plasma centrifuge, beating the Soviet record that stood for 40 years.
Now, we’re building a fusion reactor in San Francisco to achieve record power density.
BREAKING: "Speculative data center debt is being repackaged and sold to pension and retirement funds, university endowments, and other institutional investors in a manner eerily reminiscent of the subprime bubble," per AFR
Look at him, thats my Qual.
Your what?
My Qualitative! My vibe specialist. Look at him, you notice anything different about him? Look at his face.
I'll give you a hint, his name is Jayden. He won a local poetry competition in bushwick! Yeah I'm sure of the vibe.
Oh hey look, it’s another one of those amazing on-the-nose coincidences that is so totally random it couldn’t be more random if it tried to be random which it obviously did.
We need to get the shops with 10-19 employees in a position where they can produce 3x-5x their current output without requiring a corresponding increase in headcount.
Many of these shops supply OEMs & make key parts/ components that go into bigger things/ equipment.
Over the next 5-10 years, the bigger companies/ OEM’s will increase their domestic production output. That means they’ll need more of the parts/ components to assemble the finished goods.
I expect ~80% of the shops with <10 employees will disappear over the next decade. Few are willing or able to make the drastic transformation that will be required to compete & succeed in a world with widespread production automation.
The work from the 80% closing & the increased demand needs somewhere to go. Highly automated shops with 10-19 employees that employ technologies & systems built around the decades of human knowledge, will be able to ship 3x-5x what they do now. And they will do so without the administrative bloat, bureaucracy & analysis paralysis that companies with 100-150 people experience.
We need to give the shops in the 10-19 employee range, this coalition of willing, driven, & obsessed manufacturers, more support.
(Note, I’m specifically not talking about defense suppliers)
@SAndriesz WTF, ‘died by suicide’ or ‘killed by the cabal’ how many people are these pedos gonna kill? Anyone remember when they got this guy? https://t.co/z4vRzSbHDm
I am the British whistleblower who reported evidence concerning Howard Lutnick, his links to Jeffrey Epstein, and Cantor Fitzgerald and BGC to the FBI in 2020–21.
Listen to the BBC File on 4 documentary about my whistleblower story:
https://t.co/rLRDnZA4Dv
BBC News:
https://t.co/DdNNOosnKL
I later shared evidence with the House Oversight Committee, which subsequently questioned Lutnick behind closed doors.
My disclosures also contributed to regulatory action in the US and UK, including a £96m HMRC tax assessment relating to BGC’s partnership scheme. I believe the true tax exposure is substantially higher.
I also reported to the FBI and financial regulators matters involving Cantor Fitzgerald’s Relief Fund and BGC/Cantor’s annual 9/11 Charity Day.
The personal consequences of whistleblowing, and the retaliation I say followed, have been devastating for my family and me.
Mother Jones previously reported that documents I uncovered revealed a years-long business tie between Howard Lutnick, Cantor Fitzgerald and Jeffrey Epstein through AdFin, including a 2018 exchange in which Lutnick and Epstein discussed the company directly:
https://t.co/aRdn302v4i
https://t.co/aRdn302v4i
Follow me here on X and on LinkedIn. I intend to share more of the evidence and the story behind what happened.
"Humans are ingesting so much AI content that prompt language, fawning feedback and “caveman speak” are officially bleeding into real-world meetings," per WSJ