Morgan Howells 🏴‍☠️

Technology always wins

[Kimi.ai @Kimi_Moonshot]

Releasing the model weights and technical report of Kimi K3.

Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.

New model architecture: 2.5x the intelligence per unit of compute, not just more params.

Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.

Model weights: https://t.co/7m7eEg6Y0B
Tech report: https://t.co/yeu6cjpMCT
Tech blog: https://t.co/YTfiMSNM1f

[Jensen Huang @JensenHuang]

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.

AI will transform every industry, power every company, and be built by every country.

Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.

The world needs both frontier closed models and frontier open models.

https://t.co/AUKzoQ5Ikb

[Andrew Case @attrc]

To summarize: HuggingFace got autonomously compromised by a model from an American company. HF then tried to use American frontier model(s) to defend themselves, but were blocked by guardrails. HF then had to turn to open source Chinese models to defend themselves from another American company.

[Prime Intellect @PrimeIntellect]

Announcing our $130M Series A to build the Open Superintelligence Stack

Led by Radical Ventures, with NVIDIA, Intel Capital, Dell Capital, and existing investors

Train, deploy, and continuously improve your own models using our stack.

Own your intelligence.

[Tesla Owners Silicon Valley @teslaownersSV]

Engineering is magic

[Richard Sutton @RichardSSutton]

The bitter lesson in 26 words:

Don’t be distracted by human knowledge, as AI has been historically.
Instead focus on methods for creating knowledge that scale with computation, like search and learning.

[François Chollet @fchollet]

Either you crack general intelligence -- the ability to efficiently acquire arbitrary skills on your own -- or you don't have AGI.

A big pile of task-specific skills memorized from handcrafted/generated environments isn't AGI, not matter how big.

[neurosock🧠Brain Chips🦾 @neurosock]

The hippocampus is not a library, it is a simulation engine.

HPC is known for storing maps of the environment but not so known for generating planned trajectories.

This paper proposes that recurrence in CA3 is crucial to produce those plans.

Here is my toy model and notes:

[Randall Balestriero @randall_balestr]

LeJEPA: a novel pretraining paradigm free of the (many) heuristics we relied on (stop-grad, teacher, ...)
- 60+ arch., up to 2B params
- 10+ datasets
- in-domain training (>DINOv3)
- corr(train loss, test perf)=95%
Paper: https://t.co/NpfB9G1pOP
Code: https://t.co/BsK5wmNEHc

[John Carmack @ID_AA_Carmack]

The demo we are showing is an RL agent on a laptop playing an unmodified Atari with a robotic controller moving the joystick and a camera looking at the screen.

[John Carmack @ID_AA_Carmack]

The full video of my Upper Bound 2025 talk about our research directions should be available at some point, but here are my slides:
https://t.co/mkED5nnVr1

And here are the notes I made while preparing, which are more extensive than what I had time to say:
https://t.co/BiSIzIFCuf

I had managed to go my entire career without making a slide deck. People generally seem happy enough to just let me ramble on for talks, but since I am new to the research community, I made an effort here!

[John Carmack @ID_AA_Carmack]

Even the best lab demos of AR glasses today, assembled at eye-watering prices, are not the holy vision of “augmenting reality” that Mark Zuckerberg and Tim Cook are both chasing. There are deep challenges with continuous rendering to see-through displays in this form factor for all day use.

I think there is a better “computer glasses” growth path up from the RayBans than down from Orion.

Casually talking to your AI throughout the day is clearly going to be a Big Deal, and it should be an obviously tragic mistake for Meta to not make it easy for every AI vendor to participate on equal footing.

Earbuds alone will allow that, but adding a camera is a significant step change in functionality, and glasses are the right form factor.

While high quality see-through AR may not be coming to this form factor soon, there are intermediate visual outputs that could still provide novel value. Small near-eye displays like Google Glass could be valuable for focused, single task outputs, but even trivial arrangements of tiny LEDs could be used as “visual haptics” – they don’t draw pictures, but still communicate information.

I think it is critical for the devices to last all day without running out of power to encourage the changes in behavior that make them a new wave in human / computer interaction. Really lean in hard on power efficiency; find the “speed of light” for it.

This could justify the full custom silicon that I always argued was not critical-path for VR systems, but there is likely at least as much to be gained from software optimization as hardware optimization, and the co-optimization is crucial. “Magic silicon” alone isn’t going to do the job – irresponsible software will always consume everything!

A valuable hack would be to allow them to be connected to a battery pack with a very fine wire. People wore wired headphones for decades; it is clearly not beyond the pale to consider, and you would only need a trickle of current to keep the charge topped up. A non-proprietary connector would allow the market to explore all sorts of external battery form factors – pocket, pin, cap, etc.

[the tiny corp @__tinygrad__]

What is tinygrad?

tinygrad is a formalist project. It attempts to capture the full gamut of software 2.0 in a non leaky abstraction. The methods on Tensor class create a directed graph of immutable RISC UOps defining what the computation is.

Tensor is a frontend, in addition we have an ONNX frontend and a PyTorch frontend. Whether you code in tinygrad, torch, or import ONNX models, it all boils down to the same very simple UOp graph, which you can see with VIZ=1

This graph contains nothing like matmul or conv, it's just movement ops, elementwise ops, and reduction ops. Seriously, try VIZ=1.

Below that, there's a scheduler which breaks that graph up into kernels. Then we do more graph transforms on each kernel subgraph until we have code which can run on an accelerator. See the kernels with DEBUG=2.

Then we have runtimes capable of running that code. For AMD, our runtime goes all the way to the physical hardware; we are mmaping the PCIe bus and peeking and poking it. It's all in Python, but it is fast because once you have the graph compiled, you are running the same graph over and over; just ringing a doorbell.

The hope is that, similar to Linux and LLVM, we will prevent a major source of rent seeking in our AI future. By clearly and simply specifying the job, being able to precisely spec what is bought and sold, you can have a fair marketplace for compute.

By the end of the year, we should be similar in speed on NVIDIA to the existing torch CUDA backend, except without CUDA. We will also have a test cloud up where you can run jobs from any of the three frontends.

You don't want to rent a GPU per hour on a machine, you want to rent a couple FLOPS in a lambda function. That's what the OpenAI API is. Now offer it decoupled from the specific model.

[Mainak Deb @MainakDeb19]

Unlike the brain, most ANNs lack any kind of organization of units🧠

In our paper (ICLR 2025) with @mayukh091 @apurvaratan, we present TopoLoss, a simple, scalable way to add brain-like topography to AI models with minimal performance loss🧵

https://t.co/JXU3NXQkLb

[Kording Lab 🦖 @KordingLab]

Dendrites doing AI problems, so great to see the increased activity! https://t.co/63Hxqh9kHC

[Andrej Karpathy @karpathy]

I still do this most days and I think it works great. My morning brain (right after 1hr exercise and 1 coffee) is quite eager to work and I go directly to the one top priority item. The energy decreases over time and with every distracting item loaded into the context window.

[Paul Graham @paulg]

Possible heuristic for deciding what to do this year: what do you wish you'd done last year?

[Sakana AI @SakanaAILabs]

Introducing ASAL: Automating the Search for Artificial Life with Foundation Models

https://t.co/4FMqZ98CSb

Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems!

We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project.

Full Paper (Website): https://t.co/0cF28Swid6
Full Paper (arxiv): https://t.co/NnOkez0V8r
Code: https://t.co/BlZnGJK4g8

In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms!

Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life.

We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.

[Jeremy Howard @jeremyphoward]

I'll get straight to the point.

We trained 2 new models. Like BERT, but modern. ModernBERT.

Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.

It's much faster, more accurate, longer context, and more useful. 🧵

[Asimov Press @AsimovPress]

The Dangers of Mirrored Life 🦠

In a Science article published today, 30+ scientists call on the broader community to confront the serious risks of creating "mirrored" lifeforms.

They lay out their case in a compelling, 300-page technical report.

Check out our Special Report🔻