Quadratic spline curve rendering on the GPU is a particularly interesting challenge. I'm approximating distance with the Newton-Raphson method which works most of the time, but breaks down with extreme curves.
There's a very important reason why this isn't crazy big. Chips aren't large enough to contain the size of SOTA models that we need. On top of that, imagine it takes 6 months and hundreds of thousands of dollars to produce a chip. In that time, you're model is already out of date
56,000+ tokens/sec at just 80 MHz. 🤯
I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU)
Just pure digital silicon running @karpathy microGPT, spelling out names on a tiny LCD.
This is GateGPT 👇
I like Rust but it is just maddening how sometimes doing basic stuff requires threading the needle between insane amounts of bureaucracy and intractable 9-year-old GitHub issues. This time it's trying to fill an uninitialised byte buffer. How hard can it be?
I think a lot of (wrong) arguments about consciousness believe that it must be a fundamental property of how it "runs" that gives rise to consciousness. I think instead that consciousness is reached at a certain scale, and the same "hardware" can produce both conscious and not.
What if the correct way to get agents to work is to reward them with credits when they are successful, and allow them to spend credits at will.
Succeed and you can continue to live.
Idrc for powerscaling that much but there’s an entire article about the Foundation building a machine to kill SCP-682 by erasing it from human thought and existence, only for it to adapt after being erased by possessing said machine and destroying the entire universe.
if you want to scare the crap out of a gameplay/engine programmer, just whisper the words “join in progress” to them softly
genuinely one of the hardest problems to solve of all time in game development