I spent the last six months trying to deconstruct Taalas's patents. We think it can be better: 100x fewer memfetches than their bitROM, better software by better quantization than their hardware team could.
So we wrote a compiler that:
> takes a huggingface checkpoint,
> quantizes the model
> descends the weights down metal layers to RTL and GDS, do your DRC, Yosys, PEX, make the electrical waveform execute a mat-vec from your huggingface checkpoint (thanks Cambricon tech papers)
in ~7000 lines of human-readable code.
we're looking for someone with contacts with a foundry/access to 7nm PDKs or contacts at a cheap EuroPTW shuttle? I'm broke and unemployed.
@itsclivetime this is what i think is the future of perplexity per picojoule. @zerohedge@zephyr_z9 you guys wanna see 40,000 tokens/sec?
Jensen drove supply chain brilliantly at Nvidia, but many of those deals were a handshake with Jensen, not a written contract.
Without Jensen, at least some of those critical deals will be lost.
Just a random fact of trivia: When I left microsoft frustrated not being able to work on machine learning in 2013 to google, I yolo wrote in the exit survey that I would stay if I get to work with John, and now destiny and world has manifested this :)
Also as in https://t.co/EVlOugANyA, KFAC, FOOF, Muon, and most of PSGD/matrix opt is just a 2x2 choice matrix as well.
Here's a proof of @_arohan_'s Locoprop-S being basically equivalent to KFAC; note that Locoprop-M with bregmann divergence is different and requires iterations.
I installed https://t.co/tQncGXpcRY 13 years ago and averaged > 3 games/day since (15,000 games!). Even crazier: my Elo rose for the first 3 years, flatlined for almost 6, and then started rising again in the last 3 after I made a rule of not playing when drunk or before coffee.