Sergio Rus

[Alex Veremeyenko @alex_verem]

Researchers at Oxford argue that LLMs can't invent anything.

It's impossible mathematically.

The paper is called "Theory Is All You Need."

Teppo Felin and Matthias Holweg take the famous "Attention Is All You Need" title and flip it. Their argument is that AI predicts from the past, while humans reason forward into the future, and those are two different kinds of thinking.

Start with the numbers. The authors estimate a large language model trains on roughly 13 trillion tokens. A human reading at 150 words a minute would need about 164,000 years to get through that. A child hears around 20,000 words a day and roughly 36.5 million words in their first five years. It's the same task with wildly different data, and the child still ends up with language that goes far beyond anything they heard.

Their point is that the model learns which words tend to follow other words. It becomes a mirror of what people have already written. It doesn't build a theory of how the world works, so it can't step outside its training data.

The paper's sharpest thought experiment makes this painful. Imagine an LLM trained in 1633 on every scientific text ever written up to that point. Ask it about Galileo and heliocentrism. Thousands of years of geocentric texts would swamp Galileo's ideas, so the model would tell you he's wrong. It would also rate Tycho Brahe's astrology as more credible than the idea that the Earth moves, because more people had written about astrology.

Then there's flight. In 1888 the scientist Joseph LeConte looked at bird data, noted that no bird above 50 pounds could fly, and concluded humans couldn't either. Lord Kelvin, then president of the Royal Society, said he had not the smallest molecule of faith in aerial navigation.

The New York Times estimated in 1903 that flight was one to ten million years away.

Nine weeks later the Wright brothers flew.

The Wrights didn't have better data. They had a theory. They broke flight into three problems, lift, propulsion, and steering, built their own wind tunnels, and generated the data that didn't exist yet.

Wilbur wrote in 1900 that he had been "afflicted with the belief that flight is possible to man."

Every prediction machine on Earth would have told him no.

The authors call this the data belief asymmetry. Every real breakthrough starts with someone believing something the existing data says is wrong. A system trained to minimize surprise can't do that by design.

They're not anti AI. They say AI will win at routine, repetitive decisions that extrapolate from the past, which is most decisions. They're just pushing back on the idea that you should replace humans with algorithms whenever possible, which is a direct quote from Kahneman.

I use these models every day and this matches what I see. The new stuff comes from the human at the keyboard who decides the data is wrong.

LLMs don't think, you do!

[Operador Nuclear @OperadorNuclear]

Red Eléctrica ha vuelto a lanzar una señal de alarma: está detectando variaciones bruscas de tensión en la red y pide medidas urgentes para evitar un nuevo apagón. No lo dice abiertamente, pero el mensaje técnico es claro: al sistema español le falta potencia firme y síncrona (nuclear, gas e hidráulica), que aporta inercia, control de tensión y estabilidad.

Las renovables conectadas por inversores pueden cambiar su potencia en segundos, prácticamente en escalón. Esa rapidez, que en apariencia parece positiva, puede provocar oscilaciones que el sistema no siempre logra amortiguar. Antes, esas perturbaciones se disipaban gracias a las máquinas síncronas de las centrales nucleares, hidráulicas o de ciclo combinado, cuyos alternadores giran solidarios con la frecuencia eléctrica. Su masa rotante actúa como un estabilizador natural, frenando los cambios bruscos y manteniendo la tensión dentro de márgenes seguros.

Pero cada vez hay menos generación síncrona conectada. Cuando una nube cubre una planta solar o el viento cambia de golpe, el sistema responde con saltos eléctricos que pueden causar desconexiones automáticas de generación o demanda. Si eso ocurre en cadena, el resultado es un apagón.

REE no puede resolverlo por sí sola. El mercado eléctrico decide qué centrales operan, y si las síncronas quedan fuera por precio, el operador no puede encenderlas aunque las necesite para dar estabilidad. Por eso pide al Gobierno y a la CNMC que modifiquen los procedimientos de operación, limiten las rampas de las renovables y refuercen el control de tensión.

No menciona directamente la falta de potencia firme porque hacerlo implicaría reconocer su propia responsabilidad en el apagón de abril, cuando el sistema operó con una fracción mínima de generación síncrona. Pero el mensaje entre líneas es inequívoco: el sistema necesita más masa girando, más generación estable conectada para sostener la red.

La transición energética no consiste solo en producir megavatios-hora con bajas emisiones, sino en mantener el sistema en pie. Las centrales nucleares, que el Gobierno quiere cerrar, proporcionan las dos cosas. Sin potencia firme, no hay red segura.

https://t.co/YD8NsZFL4U

[Noah Jakob Rettberg 🇩🇪🇪🇺 @NoahRettberg]

"What the Hell is Going On in Spain?"

At around half past eleven today, Spain and Portugal experienced an almost nationwide blackout.
Small parts of southern France were also briefly affected but reportedly regained power quite quickly.

The Portuguese grid operator currently attributes the likely cause to weather-related oscillations in power lines, which caused the grid to collapse.

At the time of the load shedding, Spain’s electricity supply was in a very fragile state: 70% of the electricity was generated by producers without grid-synchronous inertia.

Grid-synchronous inertia refers to turbines in nuclear, coal, gas, or hydroelectric power plants that drive the generators of these plants.
The rotational speed of these turbines is synchronized with the frequency of the power grid.

Thus, the grid frequency is directly linked to the rotational energy of the turbines.
If the frequency drops, it means that the load on the grid exceeds the output of the power plants connected to it.
In such a case, the rotational energy of the turbines is converted into electricity to stabilize the grid.
The valves on the turbines respond by opening further to support the grid.
The opposite situation can also occur: if there is excess generation, that energy is absorbed as rotational energy, and the turbine valves close accordingly to adjust to the new load.
Although the rotational energy compared to the grid’s power consumption is not large, it is sufficient to stabilize excessive load fluctuations or absorb surplus generation until the valves have time to react.

This principle of grid-synchronous rotating masses has been the foundation of our power grids for over 100 years, ensuring that at every moment exactly as much electricity is produced as is consumed.

Over the past two decades, Europe has been rapidly building power generation facilities that do not use grid-synchronous turbines, but instead produce direct current, which is converted to alternating current via inverters:
Wind and solar energy.

These inverters can generate high-quality alternating current, but they cannot replace the function of grid-synchronous inertia.

The more wind and solar power is integrated into the grid, the fewer conventional power plants with grid-synchronous turbines remain.
The fewer grid-synchronous turbines there are, the less inertia there is to buffer instantaneous fluctuations in load.

If there is insufficient buffering, even small fluctuations can quickly cause the grid frequency to deviate dangerously from the ideal value of 50 Hz.

Consumers and producers alike respond to this by disconnecting from the grid for self-protection, which can trigger a cascading collapse.

What exactly tipped the balance in Spain remains unclear.
However, we do know that the Spanish grid was in a highly volatile state at the time of the collapse.

Please note:
This text is an AI translation of my orginal German language Tweet.

[Operador Nuclear @OperadorNuclear]

Algunas publicaciones envejecen muy mal, y demasiado rápido.

Ha bastado un único suceso iniciador —todavía por esclarecer— para desestabilizar la red eléctrica, desconectar buena parte de la generación solar y eólica (que en ese momento aportaban el 78% de la producción) y precipitar con ello la caída de todo el sistema.

La falta de generación síncrona —con la mitad de la potencia nuclear parada por precios bajos e impuestos abusivos, sin apenas ciclos e hidroeléctricas— ha dificultado estabilizar la red y evitar el apagón.

La generación síncrona aporta inercia y capacidad de respuesta instantánea frente a perturbaciones. Con más generación síncrona en servicio, la red habría tenido más resistencia frente al fallo inicial y habría sido mucho más probable contener el desequilibrio sin que se produjera el colapso.

Como ha quedado patente y se ha advertido muchas veces, cerrar las centrales nucleares es un suicido energético.

[Paul Graham @paulg]

You can tell just by looking at this progression that this was a labor of love for a lot of very smart people.

[Massimo @Rainmaker1973]

Anthony Jeanjean at the BMX freestyle finals in Paris #Olympic2024

[Michael Morelli @morellifit]

Europeans can be in amazing shape & live to 100+ while:

• Eating carbs 24/7
• Rarely going to the gym
• Drinking alcohol every day

But when Americans do it?

They become obese & get sick.

It took some time...

But I think I found what EXACTLY is going on:

🧵

[CSIC @CSIC]

Un estudio preliminar desarrollado por el CSIC muestra una mayor toxicidad en las bolsas compostables que en las de plástico convencional. Esta toxicidad observada en las bolsas compostables se traslada al abono durante el proceso de biodegradación.
➡️https://t.co/UIIZ7qtSIg

[Operador Nuclear @OperadorNuclear]

Alemania se come a sí misma porque está cerrando sus centrales nucleares y necesita quemar más carbón.

[Sergio Rus @sergiorus]

Hay que reconocer que es un temazo https://t.co/5a0Yv3oQ8u

[TechCrunch @TechCrunch]

🚨 LastPass says customers' password vaults were stolen in its recent data breach.

https://t.co/3CECibFKvj

[Sam Altman @sama]

ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness.

it's a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness.

[Sergio Rus @sergiorus]

Artículo interesante sobre ChatGPT https://t.co/yoNOL35g8V

[Max Roser @MaxCRoser]

If you want to reduce your carbon footprint, focus on what you eat, not whether your food is local.

The chart shows why: transport tends to account for only a small share of greenhouse gas emissions — the differences between different food choices are much larger.

[Sergio Rus @sergiorus]

Best _crescendo_ ever in music https://t.co/oXfTNjPrNy

[Cloudflare @Cloudflare]

Today, June 21, 2022, Cloudflare suffered an outage that affected traffic in 19 of our data centers. This was caused by a change that was part of a long-running project to increase resilience in our busiest locations. Here's what happened: https://t.co/Hb7lRJ2ND6

[Sergio Rus @sergiorus]

A decade later, Yeah Yeah Yeahs is coming back with new album and a new great song! https://t.co/IFnavBeCt8

[Sergio Rus @sergiorus]

@lorenzogil El sonido del protocolo de conexión era una "melodía" que más de uno tenemos grabado a fuego. Aunque en mi caso era de 56K. No sé si el sonido en ambos casos era el mismo...

[Sergio Rus @sergiorus]

Parece que mucha gente olvida que la inmunidad completa de la vacuna COVID no se consigue hasta pasadas 2 semanas...

https://t.co/jdpXukpX1I

[Sergio Rus @sergiorus]

Once again, many sites down due to an issue with a massive CDN provider. This time that honour goes to Fastly. Maybe we should drop the words "distributed" and "reliable" when talking about the Internet.