I remain convinced chat is the wrong interface for almost all serious work. Even when chat's a reasonable interface a document centric one is better. Chat is only good for quick questions / sessions.
they were pretty conservative with their paper
so here are some bold and cope potentials if it holds up at scale
> 3-4x memory reduction across the board without much quality loss
> train a small/mid sized LLMs on a single GPU
> if you can train each block independently without much comms: less all-reduce, fewer pipeline bubbles, and reduced comms overhead
> if it works on fine-tuning existing models: consumer GPUs/small clusters can fine-tune SoTA models
> if blocks are independent: partial fine-tuning gets cheaper, since you can update subsets of blocks instead of the whole model
feel free to shut me down
~ memory is a flock of birds ~
i built a hopfield network and taught it the alphabet - then watched it remember in real time by adjusting the temperature.
no neuron has the whole picture. the memory is distributed across every neuron’s connections.
"I do not think a chatbot is the right interface for travel or e-commerce." - @bchesky
"I think the future is not apps. The future is agents, but I don't think they're going to be text-forward. I think they're going to be really rich user interfaces."
"Imagine using iMessage to do everything, when in fact every other app has a unique interface."
"With e-commerce, you want a very rich user interface. It would be agentic. You can have a conversation with it, but the point is that it has to be more visual."
Today I got the "you speak English very well" from an older Swedish-American lady, upon learning I'm from Iran.
She does not speak English very well and lives on "mount La Jolla [sic]" :-)
I was tempted to find it slightly offensive, but she's right, I speak very well indeed.
The confounding factor is that virtually every big company is overstaffed by 2-4x and has been for decades. AI is the catalyst/excuse to finally fix that. Of course nobody wants to say this out loud.
Deepseek v4 running locally, particularly @antirez 's work, is a very important moment in time people are sleeping on. Once local / hackable becomes good enough ecosystems of super interesting work will emerge. Future hardware systems will spec to enable this.
The layoffs are not about AI, they're cost cutting. AI is a convenient excuse, and now it's fashionable so if you're not "laying off because of AI" you're seen as not embracing the cutting edge. I will (theoretically) write more intelligently about this at some point.
I'm seeing a lot of people in enterprise settings using the word "Claude" as a somewhat generic term for AI agents, as in "let's connect Claude to it" to mean let's connect an AI agent to it.
I was thinking how strange it is that Finnish sounds exactly like French, and then I realized it was just a large group of French tourists visiting Finland.