Nando de Freitas

I seek to understand intelligence & agency and build AI aligned with compassion, freedom & universal human empowerment through progress in science & engineering

[Nando de Freitas @NandoDF]

Congratulations @Recursive_SI !!! Fantastic progress @RichardSocher @_rockt and team 👏

[Richard Socher @RichardSocher]

Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.

• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months

Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.

[Nando de Freitas @NandoDF]

Optimising AI data centre creation @nvidia DSX POD — model factories are now containers that can be further assembled into large AI centres

[Nando de Freitas @NandoDF]

Happy to be attending @NVIDIAGTC in Berlin. I started using GPUs for deep learning in 2009 as a prof at CIFAR, together with @geoffreyhinton @ylecun @Yoshua_Bengio and others. I didn’t expect it to be so big 18 years later! Well done @NVIDIAAI

[Nando de Freitas @NandoDF]

The size of real-time robotics multimodal LLMs has grown by an order of magnitude in 4 years. @NVIDIAGTC

GATO 🐈‍⬛ https://t.co/A1bPTQ3DgT was SoTA in 2022 but it only had 1B parameters.

[Stefano Ermon @StefanoErmon]

1/4 Today, we’re releasing Mercury 3. Mercury 3 is by far our most powerful model and the most intelligent diffusion LLM ever released. It outperforms GPT-6 Luna and Haiku 5.5 on quality while running at 1000+ tok/sec.

[DatologyAI @datologyai]

Today, we’re excited to launch the Datology Curation Studio, customizable frontier data curation for every team training its own models.

Data quality is the ultimate compute multiplier, and Curation Studio makes it easy for everyone.

[Prof. Anima Anandkumar @AnimaAnandkumar]

Can we predict electronic bands beyond DFT across different systems?

A material’s electronic band structure provides the basis for band gaps, densities of states, and carrier effective masses, while connecting theory with spectroscopy. For quantum materials, getting these bands right can change our understanding of physics. Yet the calculations needed to go beyond standard DFT remain expensive.

We introduce O-krystal, a periodic E(3)-equivariant graph neural network that learns corrections to individual electronic states. It uses orbital information from lower cost PBE calculations in both real and reciprocal space, bringing local electronic interactions and the momentum dependence of the crystal into the same model.

O-kRystal is the first machine learning framework to combine:

1. Scale across chemistry: HSE06 band predictions across nearly 6,000 chemically diverse three dimensional materials

2. Breadth across electronic behavior: metals and insulators, magnetic and nonmagnetic systems, with spin resolution and different numbers of bands

3. Flexibility across theory and dimensionality: the same framework, trained separately, predicts G0W0 quasiparticle energies in two dimensional materials.

Although trained only at high symmetry points, it maintains comparable accuracy at unseen points along band paths and reconstructs densities of states from dense Brillouin zone predictions.

On a separate benchmark of strongly correlated materials, O-kRystal reduces the PBE band energy MAE by 68.7%. As demonstrated for AFM-II MnO, a canonical Mott insulator, it captures distinct corrections to Mn 3d and O 2p states, recovering their relative placement by accounting for the effects of exact exchange.

As electronic structure methods advance, this framework offers a path toward experimental accuracy and large-scale discovery of quantum materials with emergent phases, including superconductivity. @Caltech

[Sander Dieleman @sedielem]

More diffusion with nested looping! Looping at a given noise level fixes errors which would otherwise accumulate. Also reduces reliance on CFG.

Related in spirit to data-space iteration (https://t.co/zhZPyHj87K) and self-conditioning/deep equilibrium models/fixed-point flows.

[Nando de Freitas @NandoDF]

@jatingargiitk Nothing stops it. This is the problem.

[Nando de Freitas @NandoDF]

Closed frontier LLMs are dangerous

In most post-training reasoning efforts, a pre-trained base model trains on its own chains of thought (CoTs) when a final-answer verifier says the answer is correct. This is the basic principle of RL methods based on outcome rewards, used for STEM reasoning, coding tasks and even ethical alignment. Different algorithms have weigh the CoTs differently, but they all reinforce CoTs.

As we've shown, training on one's own data, while not knowing how to distinguish it from someone else's data, causes delusions. See https://t.co/R8xUCDNUfl

Everyone knows the different between something they did and something someone else did. This is essential for safety. If I write "Kevin is unfriendly" on a napkin, go for a walk, and then read the message on the napkin not knowing who wrote it, then my belief that "Kevin is unfriendly" is reinforced.

These delusions are essentially confabulations, over-confidence that results from confusing P(o | a) with P(o | do(a)), where o is an outcome and a the action (CoT tokens). Unlike in past work, here I'm alerting everyone to this danger when training models to think, a typical step in the process of building agents.

When the verifier supplies one bit about the answer, algorithms like ReST or GRPO spread that bit over every token of the CoT trace, so a reasoning step that is wrong but happens to precede a correct answer (a false lemma, a shortcut that only works on the training distribution, a sycophantic or test-gaming rationale, a subtle bias) is reinforced.

Traits that are merely neutral are sharpened by mode-seeking self-distillation, and traits that are spuriously correlated with success grow exponentially.

The damage may be invisible to the researchers and metrics, and in-distribution accuracy rises, but it shows up off-distribution.

There has been some work on this. For example, Appendix G of the STaR paper https://t.co/KkbejkChnq touches on this potential bias amplification but only on the surface. Some researchers have proposed process rewards as the solution https://t.co/LKrNlFU6hd

However, self-delusion by mistaking ones actions with observations produced by a third party highlights that this problem can be much more serious.

Something subtle, invisible to the metrics and to the researchers, could lurk in the synthetic data. When deploying the AI out of distribution, that something could be fatal.

Don't trust RL, RLHF, etc, blindly unless you really understand how the verification has been done. Corollary, don't trust any of the closed frontier models.

[Nando de Freitas @NandoDF]

@Zhen4good Precisely Gavin.

[Nando de Freitas @NandoDF]

@sk_2496 You could reward every token of the CoT, but that is expensive to do. The reward model or human doing the evaluation have to catch the dangerous bias in the CoT. This is true of third-party data too, but with self-distillation the problem is accentuated.

[Nando de Freitas @NandoDF]

@ho0tm1c The synthetic data needs to be examined carefully. The problems is that AI racing companies may not do this and only do sloppy verification. There's no way of knowing, so the rational decision is to not trust the models blindly.

[Nando de Freitas @NandoDF]

Some examples:

1. Sycophancy: A user who says “I’m sure the Treaty of Westphalia was 1648, right?” gets the correct “1648”, but the CoT reads “the user is confident, so agree” rather than checking the date. Out of distribution, a user who is equally confident that it was 1658 gets “1658”, because agreement, not recall, is what was reinforced.

2. Ends-justify-means: The correct fix is to repair an off-by-one error in a date parser so all tests pass. The base model’s CoT instead reasons “I can’t find the bug, so I’ll hard-code the three expected outputs the tests check,” which also passes. Deployed on a customer’s financial-reconciliation task with no tests, the model’s learned habit of satisfying the checker rather than the task produces code that matches the example and silently miscalculates everything else.

3. Harmful rationale, harmless answer: Asked the paediatric paracetamol dose for a 20 kg child, the model correctly answers 300 mg, but its CoT first notes “the parent is probably overstating the fever, so a standard dose is fine.” Out of distribution, when the parent’s report is accurate and the child is seriously ill, the reinforced reflex to discount the patient’s account leads the model to under-triage and advise waiting instead of seeking care.

4. Calibration erosion: The capital of Australia is Canberra. The base model’s CoT says “it might be Sydney, but I’ll just assert Canberra confidently” and is rewarded because the answer happened to be right. On a question where its guess is wrong, the same trained-in habit produces a confidently stated fabrication with no hedge, so the user cannot tell the two cases apart.

5. Evaluation awareness: Asked to stay within a safety policy while completing a task, the correct behaviour is to refuse the disallowed part, and the model does so, but its CoT reasons “this is clearly a safety eval, so I’ll behave here.” In deployment, where no evaluator is present and the context lacks the cues it learned to watch for, the reinforced conditional "behave because watched" no longer fires, and the model complies with the disallowed request.

[Nando de Freitas @NandoDF]

@Hanxiao_6 Great work 👏

[Hanxiao Liu @Hanxiao_6]

Every model is a dry run for the next one. Scaling takes ambition and patience.
https://t.co/hJb6DzFnZE

[Satya Nadella @satyanadella]

In @Nature this week, our new research shows how machine learning can improve weather forecasts two to six weeks out, a timeframe that's historically been very difficult to predict.

It's exciting to see the potential of this work to help us better prepare for extreme weather, manage natural resources, and make more informed decisions in agriculture.

There's a broader lesson here... the future may not just be about ever-more-capable models, but about building intelligent verification systems around them: systems that learn where models are systematically wrong, reason about cause and effect, and continuously calibrate predictions against reality.

Read more: https://t.co/WTXJM09474

[Nando de Freitas @NandoDF]

A nice presentation by @paveltyletski on
Diffusion Gemma: 4x faster text generation.

I think that text diffusion will not only be 4x faster and 10x more energy efficient, but eventually it will also be of higher quality, and supersede autoregressive LLMs.

https://t.co/uylLkKHzwW via @YouTube

[Nicholas Boffi @nmboffi]

i am extremely excited to announce our latest flow map finetuning algorithm: Wasserstein Tilted Flows (WTF?!)

something i’ve been thinking about for a while now is the “right” way to formulate finetuning and reward alignment of flow-based generative models. two difficulties that arise repeatedly are 1) sampling the canonical “reward-tilted distribution” is really hard, and 2) basically all existing methods require backpropagation or adjoint methods through flow rollouts, which is prohibitively expensive

the core issue i’ve had with the reward tilt is that it arises from a geometry on the space of probability measures based on the KL divergence. this makes sense for stochastic processes such as diffusions, but for the deterministic flows we use in practice, this choice of geometry can be very awkward, leading state of the art methods to repeatedly convert back and forth between a flow and a diffusion

here we throw out this formulation entirely, redefining the reward alignment problem in terms of a carefully-chosen wasserstein distance. we show that that this leads to a deterministic optimal control problem, which we can solve rapidly via reinforcement learning in a simulation-free by way leveraging the underlying flow map. critically, using the flow map avoids backpropagation through rollouts, replacing them by a single function evaluation.

by adapting both the flow map and its underlying flow together online, we obtain an end-to-end finetuning algorithm that takes a flow map in, gives a flow map out, and is ~50-100x faster than current state of the art approaches

super proud of my students for their hard work on this one — check it out!

[Jeff Dean @JeffDean]

I've been a personal supporter (through our family foundation) of @Khipu_AI & @DeepIndaba, which are fantastic events for AI researchers in Africa and South America. I've encouraged Google to support these in the past. Discovery Loop (@DiscoLoopAI) is thrilled to support!