For the people asking if this is just a video... nope!
Here's me playing it on my Macbook.
It's only been ~13 hours so far, so expect it to get a LOT better :)
@DeepSeekHarness Hello DeepseekHarness team ,was is not happening to your harness,I just migrated to it and in the benchmarks it is so down bad https://t.co/wmfIaInRIA
https://t.co/UStDmf5MNx chief scientist Jie Tang just revealed the training pipeline behind the upcoming GLM-5.5 (32B, 122B, and 256B) during a lecture at Tsinghua University today.
Instead of relying on public datasets, their pretraining pipeline pulls from six separate internal engines:
- Web: 1 trillion pages crawled at ~10B pages/day with PB-scale template clustering
- Documents: 2 billion scientific PDFs parsed through GLM-OCR (4x faster with speculative decoding)
- Books: 50 million books with layout reordering and garbled-text filtering
- Code: 500 million repositories turned into executable tasks with real tests and issue descriptions
- Video: 10 billion videos for long-sequence world knowledge
- Audio: 300 million hours rewritten via ASR into trillions of tokens
As a coder for 35+ years, it hurts to say this, but:
the real sloppy code was written by humans, not AI
AI is having a blast hacking that code left and right, right now
my spidey senses are telling me the labs are, as we speak, using unguardrailed versions of their unreleased models to systematically reverse-engineer ALL of the internetโs software, apps, and games, then feeding the reconstructed source code straight into their training pipelines while simultaneously building guardrails to ladder-pull the rest of us
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7ร faster and full attention 72.4x faster.
With @kubicek_ales@caciolai@PengLiangzu@SukritSumant@Ben_Hoov@FraserGreenlee@RMBattleday@radim_urban@thoefler@jcrwhittington
@LLMSherpa I respect you Guys,Shout out to you all from Kenya...Saving a whole generation from being a permanent underclass in the new era of AI is peak Humanity.I cant imagine a world without Torrents equivalent for AI models
๐ซก๐ซก๐ซก
Yann LeCun and Randall Balestriero just Proved the Platonic Representation Hypothesis, and found a better way to achieve it.
The Platonic Representation Hypothesis suggests that AI models trained on different modalities, like images and text, eventually learn similar representations of the world. But this paper shows that how models are trained matters more than simply scaling them up.
So they mathematically show that, under certain conditions, two JEPAs trained completely separately can learn essentially the same underlying representation of the world. One learns from different images of the same scene, while the other learns from different descriptions of that scene, without ever being trained on image-text pairs.
Their approach achieves the same image-text alignment with 42x fewer training samples than next-token prediction. A 109M-parameter text JEPA even outperforms LLMs up to 72B parameters on cross-modal alignment benchmarks.
https://t.co/3j1enq9qF7
i've been feeling a bit COLD SHOULDERED on UV laser PCB fab (i.e. nobody is funding me to do it) so to share the culmination of a 1yr+ of substrate testing:
Double-sided aluminum copper clad (ceramic dialectric)
Common, cheaply available, and there is no issue with aluminum contact even during ablation. It is much better than FR4 for UV laser ablation