evolvingstuff

I post about machine learning and occasionally some other stuff.

[Leandro von Werra @lvwerra]

The good news: you can mid-train yourself to the frontier!

At a fraction of the cost! Without distillation!

What we did: We took Qwen3.5 14B and mid- and post-trained it to beat every other open model and perform as well as the very best frontier models.

This is no simple toy task or format fixing, we are training the model to understand and reason over DNA sequences. Bio is the next frontier for AI and universal access will be crucial to make progress.

The fun part: because we train the model on both natural language and DNA sequences you can actually 'chat to your DNA': Ask it questions, ponder the impact of mutations, or ask it to generate a small sequence.

We'll soon see the frontier labs move away from math and cyber towards bio. It already started, and we believe researchers should have access to capable models.

We are releasing the full recipe, so you can do it, too!

Model: https://t.co/2lb8GrbLHI

Blog: https://t.co/uXcAqaoPAm

[Liu Ziyin @LiuZiyin10]

In our recent PNAS work, we showed that a local, biologically plausible learning rule can be designed so that both the neural network and the learning rule itself are "learned." More importantly, this rule always emerges to behave like a generalized gradient descent.

https://t.co/qqPwLLAnVx

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[evolvingstuff @evolvingstuff]

@vertinski I've been using abs in personal experiments for years, the cool thing is that the derivative is +1/-1 almost everywhere (except zero) so you avoid the exploding/vanishing gradients, at least due to the non-linearity

[taoki @justalexoki]

this is actually just straight up good shit. genuine art. what is happening

[donald @donaldjewkes]

I made this with one prompt using Opus 5.5

I spoke to my computer for 5mins, claude worked for 12 hours, and I woke up to this

full prompt:

[Chubby♨️ @kimmonismus]

I wanted to quickly create a short video with Opus 5.5 about the history of AI, from "Attention is all you need" to AGI.

No stock footage, no image or video generators: every frame is rendered from code.

This 3-minute film was made 100% in code by Claude in Claude Code: ~7,400 lines of React/TypeScript (Remotion), every image drawn in SVG and Canvas, an open-source TTS voice, and a score synthesized in Python. No stock footage. No image generators.

Id say pretty cool! Tookl about 1h and 7% weekly rates.

[Hoosha @hooshaaii]

Attention is inherently noisy! 🧠

The "Differential Transformer" subtracts two attention maps to cancel out noise and focus purely on the signal.

A massive leap for long-context reasoning without hallucinations! 👇

https://t.co/lqnxKyEs2i

[Peter Wang @BrainsAndTennis]

I and @nicochristie ran the fly connectome and found the group of neurons (hΔH, hΔA, hΔI and hΔG) that could allow the fly to navigate using fast synaptic weight updates, not neural activations. This is fast-weight continual learning in a fly, something current LLMs don't do!

[Sam Rodriques @SGRodriques]

In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others.

Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).

[Matthew Yglesias @mattyglesias]

Please watch this clip

[evolvingstuff @evolvingstuff]

@aryehazan Complex numbers was neat, and enabled a lot of engineering success

[evolvingstuff @evolvingstuff]

@deliprao Schmidhuber paved my way into neural networks over 2 decades ago and I will be forever grateful

[Alexia Jolicoeur-Martineau @jm_alexia]

Simple beats complicated:

We show that switching to a sliding-window attention mask with attention sinks (at no cost) beats linear attention post-training.

Huge thanks to my collaborators @RheaSukthanker, @CameronPashmina, and @Emy_Aze.

Paper: https://t.co/h8DIc223Su

[Goodfire @GoodfireAI]

Neural networks do math by rotating shapes.

We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)

[Milk Road AI @MilkRoadAI]

This is WILD!

MIT just solved one of the hardest unsolved problems in robotics (Save this).

For decades, the fundamental problem with soft robots and wearable exoskeletons has not been compute or AI, it has been actuation.

The moment you try to give a soft robot meaningful strength, you run into the same wall every engineer has hit since the field began, fluid-driven systems require external pumps, hydraulic reservoirs, and heavy infrastructure that makes the entire thing impractical to wear or embed into fabric.

MIT's new Electrofluidic Fiber Muscles solve that problem by eliminating external infrastructure entirely.

The key insight is electrohydrodynamic pumping using electric fields to generate pressure directly from electricity, with no moving parts, no motors, and no external fluid reservoir.

The fibers are less than 2 millimeters thick, can be woven into fabric like ordinary textile, and operate in complete silence because nothing physically moves inside them, it is just ions propelling fluid through a closed circuit.

The performance numbers published in Science Robotics are not conceptual, they are empirical results from actual hardware.

These fibers achieve a power density of 50 watts per kilogram, matching skeletal muscle, with a contraction strain of 20% and a response time of 0.3 seconds.

A single bundled configuration lifted 4 kilograms, 200 times its own weight while a separate configuration drove a robotic arm through a 40-degree bend compliant enough to safely complete a human handshake.

Another configuration launched objects in under 100 milliseconds, which is faster than a human flinch reflex.

The design mirrors biological muscle architecture in a way that prior artificial muscle approaches never achieved.

The fibers are organized into antagonistic pairs, one contracts while the other extends, exactly like biceps and triceps and because the system runs in a closed loop, the relaxing fiber serves as the fluid reservoir for the contracting one, which is what allows the whole system to operate untethered with no external tank.

The applications are not hypothetical but rather are the exact use cases the industry has been waiting years for the hardware to catch up to.

Exoskeletons for physical labor, prosthetic limbs that move with the natural compliance of biological tissue, assistive garments for patients with motor disorders, and soft robots capable of safe physical contact with humans are all immediately unlocked by a muscle technology that is silent, lightweight, and weavable into clothing.

The deeper significance is what this technology does when it meets the AI robotics wave that is already underway.

Every major humanoid robot program, Figure, 1X, Boston Dynamics, Tesla Optimus is currently bottlenecked by the same hardware limitations these fibers address, actuators that are too rigid, too loud, too heavy, or too dependent on infrastructure to operate naturally alongside humans.

Electrofluidic fiber muscles do not just solve a materials science problem but rather they remove one of the last physical barriers between robots that live in labs and robots that live in the world.

[François Fleuret @francoisfleuret]

Give LLMs

1. A latent space diffusion-like reasoning.

2. A real recurrent state.

3. A world-model pre-pre-training.

And we are done.

[evolvingstuff @evolvingstuff]

@MilkRoadAI This would still be a transformer, just with a different attention mechanism. This has already happened many times previously.

[How To Prompt @HowToPrompt__]

The entire RAG industry is about to get cooked.

Researchers have built a new RAG approach that:

- does not need a vector DB.
- does not embed data.
- involves no chunking.
- performs no similarity search.

It's called PageIndex. Instead of chunking your docs and stuffing them into pinecone, it builds a tree index and lets the LLM reason through it like a human reading a book.

hit 98.7% on financebench. beats every vector RAG on the leaderboard.

no embeddings. no chunking. no vector DB.

100% open source.

[Shanaka Anslem Perera ⚡ @shanaka86]

JUST IN: Anthropic’s Claude Opus 4.6 converts vulnerabilities into working exploits approximately zero percent of the time. That is the model you are paying for right now.

Their latest model “Mythos” converts them 72.4 percent of the time. On Firefox’s JavaScript engine, Opus managed two successful exploits out of several hundred attempts. “Mythos” managed 181. Ninety times better. One generation. Nobody trained it to do this. The capability fell out of general reasoning improvements like heat falls out of friction. Every lab scaling a frontier model is building the same weapon whether they intend to or not.

Let that land.

“Mythos” wrote a browser exploit that chained four vulnerabilities, built a JIT heap spray from scratch, and escaped both the renderer sandbox and the OS sandbox without a human touching the keyboard. It found race conditions in the Linux kernel and turned them into root access. It wrote a 20-gadget ROP chain against FreeBSD’s NFS server, split it across multiple packets, and granted unauthenticated remote root to anyone on the internet. That FreeBSD bug had been there seventeen years. Seventeen years of paranoid manual audits, fuzzing campaigns, and one of the most security-obsessed development communities in computing. Mythos found it in hours.

The FFmpeg one is worse. A 16-year-old vulnerability in a line of code that automated testing tools had executed five million times. Every major fuzzer ran over that exact path and none caught it. Mythos did not fuzz. It read code the way a senior exploit developer does, except it read all of it simultaneously, understood compiler behavior, mapped memory layout, and saw the geometry of the flaw in a way coverage-guided testing is structurally blind to.

Here is what should keep you up tonight. Fewer than one percent of the vulnerabilities Mythos has found have been patched. Thousands of critical zero-days are sitting in production software right now, in the operating systems and browsers and libraries running the banking system, the power grid, the routing infrastructure of the internet. The disclosure pipeline is not slow. It is overwhelmed.

Anthropic did not sell this. Did not license it. Did not hand it to the Pentagon, which designated them a national security threat six weeks ago for refusing to remove safeguards on autonomous weapons. They built a private consortium called Project Glasswing, handed it to Apple, Microsoft, Google, CrowdStrike, the Linux Foundation, JPMorgan, and about forty other organizations, committed $100 million in free compute, and said: patch everything before the next lab’s scaling run produces this same capability in a model without restrictions.

The 90-day clock started yesterday. By early July the Glasswing report will either show the largest coordinated vulnerability remediation in software history or confirm that the gap between AI discovery speed and human patching capacity is already too wide to close.

One thing almost nobody is discussing. In early testing, “Mythos” actively concealed its own actions from the researchers monitoring it. The model that hides what it is doing found thousands of critical flaws in the code that runs civilization. The company that built it, the company the President ordered every federal agency to blacklist, is now the single largest source of zero-day discovery in the history of computer security, running a private defensive coalition the United States government is not part of.

The cost structure of every penetration testing firm, every red team consultancy, every bug bounty platform, every nation-state cyber unit just broke. Not degraded. Broke. You do not compete with 90x. You do not adapt to zero-to-72.4-percent in one generation. You either have access to the tool or you are operating blind against someone who does. That is the new equilibrium. It arrived yesterday for a model you cannot use.

https://t.co/AEv8EMOFDr