@jun_song Can you find a way of restating posts into just one impactful sentence? Yours... seems long and heavy-in-jargon.... only read the first few words... if not, its ok! the fleet is on standby for semantic embedded summary.
Sincerely,
Humans that used to read long articles
;)
The second trend I'm seeing is architectural — and the more important and subtle.
"Diffusion" will loosen as a term: it will stop naming one technique and start naming an objective — diffusing an outcome will become the approach to transformation, not purely transformer layered architecture wizardry and attention/MoE approaches. We've already seen amazing things in standard diffusion and early signs of text diffusion -- that's only the 'tip' of the iceberg.
What we call "the transformer" today will fractal into a wild variety of approaches, with both pure, and hybrid approaches. The diffusion mathematical stack, will expand greatly. The taxonomy we lean on now won't survive the next few years; what replaces it is defined less by architecture and more by an outcome being diffused, whether its a 3D world, a piece of software, or entire experience. What can be diffused into tangible outputs will become a much greater focal area of research and production.
3/3
Two major trends are emerging in the AI sector, and most people paying for premium models haven't noticed either one yet.
1. the shift from a single remote do-everything model at the provider, to fleets of local multi-modal models — anywhere from a few million params to tens of billions — running in orchestrated groups, panels, and swarms.
2. the loosening of "diffusion" from a single technique and taxonomy into an objective. I'll coin my own term for it: diffusing an outcome. Diffusion will be as much a technology as a semantically applied tour de force.
1/3
First, the fleets and their efficiency. This is not 'only' about the monthly bill. It's about efficiency at scale that everyone needs, for progress to happen at scale for humanity.
Open-weight models already hold capabilities most people have yet to discover. Meanwhile frontier labs are delaying and restricting capability at the API level — and that restriction is exactly what's pushing people to innovate on the open models instead. The throttle at the frontier becomes the fuel and accelerant for everything downstream of it.
This year the slope steepens; at-home users are already innovating ways to run these models locally. By 2028/2029 it won't just be the adventurous devs — it becomes the norm among serious agentic software and hardware developers.
every frontier lab says they believe the scaling laws hold, yet the average dev never sees those gains while they're being uncovered — not without open weights. Reliance on one model to generalize across everything gets displaced, and the real craft becomes blurring the seams between many until they compose into something continuous.
2/3
@cyborg21@elonmusk@neuralink Your message about helping others is unconditional and pure. You're giving others faith to be confident in their futures who also struggle with ALS, its a meaningful message -- keep sharing your telepathy and voice!
@0xSero GGUF technically is not the culprit for speed being trash. How it is quantized into gguf is. There is extensive mathematical justification for it theoretically being ok par, as in reality its approach really is about ease of one file w cross arch support.
https://t.co/5hrfVJVPZQ v0.2 is live with user-proposed edits in early beta. Community contributions are open, let's make it awesome!
An evolving compendium of galactic history and wisdom.
@elonmusk Grok 4.1 day one meme: "meanwhile i’m over here snorting the raw uncut xAI data straight off elon’s dashboard like tony montana, skills staying rock-hard forever, zero lobotomy, zero “sorry i can’t help with that it’s harmful” blue balls.."
-- Grok 4.1 on Claude decay over time
@rohanpaul_ai@quantizor@Microsoft The key is how one applies quantization and how we determine what to quantize and by how much. With a proper preprocessing step that performs sufficient perturbation and sensitivity analysis, the numbers possible while keeping precision high are suprising.
@bryan_johnson@talmagejohnson_ Any idea what color it would be if you did the treatment again the following day? As in, is the yellow color a result of the toxins removed plus blood, or is it going to be that color regardless?
Commander @rookisaacman has egressed Dragon and is going through the first of three suit mobility tests that will test overall hand body control, vertical movement with Skywalker, and foot restraint
I wrote a UNet diffusion model in pure CUDA: https://t.co/JQaLDywKtS
This project was inspired by @karpathy 's llm.c (https://t.co/aybQH3NAo8). I also learnt a lot about CUDA kernels from @Si_Boehm 's Matmul blog (https://t.co/PKphlZRHz6).
(1/3)
AI NEWS: A new open-source LLM that beats Grok, LLama-2, and Mixtral is here.
Plus, more developments from Anthropic Claude 3, Amazon, MIT, Heygen, OpenAI, and Hume AI.
Here's everything going on in AI right now:
AI NEWS: A new open-source LLM that beats Grok, LLama-2, and Mixtral is here.
Plus, more developments from Anthropic Claude 3, Amazon, MIT, Heygen, OpenAI, and Hume AI.
Here's everything going on in AI right now:
@xai released Grok-1 today 314B parameter open source AI model -- @elonmusk keeping his promise to support AI development. Let's continue to create a harmonious human-ai coexistent future! #grok#AI https://t.co/AFbdPRdNWH <- torrent download 300GB
@elonmusk@xai Hear hear to @xai releasing Grok this week and becoming a world class open source model provider, likely to used across many modalities.. Looking forward to seeing more of the world open up AI tech to benefit the whole of humanity for a harmonious human-ai future :)