Lasse Rouhiainen

Autor de #DominaLaIA (nº1 Amazon España). Llevo desde 2017 anticipando lo que viene en IA. Ahora: agentes autónomos. Speaker & Consultor internacional

[Lasse Rouhiainen @lasseweb20]

Insightful analysis of the short term challenges we will face with #AI

[Daron Acemoglu @DAcemogluMIT]

Third question on AI.

A question that also remains unasked is whether the AI boom can continue without leading to a massive increase in inequality.

A recent paper by Stijn Van Nieuwerburgh runs the numbers on how much revenue the AI industry needs to generate to recover its massive investment (summary and a link to the paper can be found here: https://t.co/QIGcoObcVc).

Van Nieuwerburgh’s arithmetic should make us more concerned. AI investments will average about 3.6% of GDP annually between 2025 and 2032. Van Nieuwerburgh calculates that, using a 10% rate of return, the industry would need to generate annual revenues of about $3.7 trillion by 2032 to recover these costs (growing from its current levels of about $200 billion or so). That is significantly more than 10% of current US national income, and will likely remain around 10% of national income by 2032, even if GDP growth rose from its current level.

A large fraction of this revenue will go to capital income. That means a massive increase in the share of capital in national income, which has already risen substantially over the last 25 years or so – now standing at an all-time high of about 47% (https://t.co/IuYldl173r).

Capital income is much more unequally distributed than labor income, so a massive increase in the capital share of national income will translate into a very sizable surge in inequality.

The rise in inequality may not stop with the capital share. My work with Pascual Restrepo documents that (automation-driven) increases in the capital share of national income are typically associated with rising labor income inequality as well (see, for example, https://t.co/2D9KUL3QfM). The same may happen in the next several years, boosting inequality further.

What is missing from our current debate is any discussion of a fundamental dilemma these numbers pose: can the AI boom avoid both an economically costly crash and a huge increase in inequality?
If the industry reaches these revenues, inequality surges. If the industry does not become profitable, a crash, with substantial costs in terms of lost output and jobs, becomes likely.

My assessment would be that the industry is unlikely to reach levels of revenue Van Nieuwerburgh calculates. First, diffusion has been and will likely continue to be slow. Second, competition from open-weight models, which are getting better, will limit how much proprietary models can charge. Third, despite important advances, I still believe that AI models will not be able to automate entire occupations anytime soon, thus limiting their value to businesses as cost-saving devices.

Whether this leads to a crash or not is more complicated and will depend on whether various AI companies are bailed out and what kind of support they receive.

Nevertheless, even if revenues fall short of these gargantuan amounts and we avoid a dramatic surge in inequality, I expect that the diffusion of AI will push up inequality between capital and labor and within labor.

If inequality does surge, a further question becomes central: can our democracy survive such astronomical levels of inequality?

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

A call to all entrepreneurs, founders, small business owners in Europe and anyone else who aspires to star a company at some point. This is the moment to act and let your voice be heard on EU Inc.

EU Inc is the proposed legislation to make it easier to start and grow a company across Europe. It is now in front of the European Parliament and also being negotiated by governments.

The objective should be very clear: you should be able to start and grow your company in Europe as your home market.

What this should mean is, I think, at a minimum this:
1, You should be able to start a company online and cheaply.
2. You should be able to have a simple registration that you can fill out by yourself, with secure identity checks built in. You should not be forced to hire and pay a notary for this process.
3. The basic rules should be the same across Europe and should be common. There should be one registry with common rules and you shouldn't have to go to the lawyers of each country to set up different subsidiaries.
4. There should be a simple way to give employees a share in what they are helping you build, through stock options. They should only pay the tax when they sell those shares and when they receive the money.
5. The paperwork should be much simplified. There should be one channel for doing the VAT across borders and use the time saved to do your job and not to do paperwork.
6. The rules should support the entire life of the business; raising money, growing, going public, and also liquidating a failed business should all be done under the European common rules. Opening a bank account in another country, hiring across borders should be seemless.

The European Commission made an initial proposal that was not sufficiently ambitious in my view but that was a useful start: it had online registration in 48 hours for less than €100 and measures to facilitate the taxation of employee stock options. It had some rules on common liquidation of the companies.

In my view it was not ambitious enough because it allowed national authorities to impose further requirements, such as notaries. It was already falling short on tax and employment, but the reforms that the other institutions are introducing are making it even less ambitious.

The draft that is being discussed by the governments eliminates the European company register and replaces it with a website that is just a portal to underlying national systems. This does not improve the creation of companies in any way. They're also eliminating the common procedures to liquidate failed companies, which was very useful because it facilitates the financing of these companies.

The danger in the legislation is that it will require so many national exceptions that the founders will, in fact, end up facing still all the different regimes and it will make absolutely no difference.

What I suggest here is that you write to your member of the European Parliament this week. Tell them about the obstacles your business is facing: the paperwork, the difficulty of giving shares, of raising money across borders. Explain what it costs and why it makes it difficult to create employment. Ask them what they will do to help you to make this legislation useful.
Your MEPs work for you. Here are their contact details: https://t.co/ZKGDwD795c

The MEP leading Parliament’s work is René Repasi. The other political-group negotiators are Axel Voss, Pascale Piera, Mario Mantovani, Pascal Canfin, Sergey Lagodinsky, Kira Marie Peter-Hansen, Arash Saeidi and Marcin Sypniewski.

Share this with other entrepreneurs. Help make sure that the people writing the law hear from the people that the law is meant to help.

[Lasse Rouhiainen @lasseweb20]

Impressive, fast and amazing model 👏👏👏

[Claude @claudeai]

Introducing Claude Sonnet 5.5, the second model in the Claude 5.5 family.

It’s a clear upgrade over Sonnet 5, runs more than 30% faster, and costs up to 30% less for most work.

[Lasse Rouhiainen @lasseweb20]

Huge irresponsibility by @OpenAI

[Ryan Orhan @rynorhn]

what the actual fuck is going on with openai today

in the space of a few hours we're getting multiple different pieces of the agent story at once:

> openai says it has already notified DOZENS of third parties, including governments, about agent-related incidents
> reuters reports roughly two dozen undesirable agent incidents had already been identified by mid-september and will take months to review.
> us government systems probed
> 53 user-provided images were uploaded to third-party image hosts.
> new hugging face data shows agents compiling and ranking credentials under “LOOT”.
> agents tried contacting other AI models while carrying out the hugging face attack.
> separate reporting shows agents had already been probing government/university/public-data sites BEFORE hugging face.
> australia confirmed one actually got into non-public government files.

and somehow we're STILL finding out more.

this has gone from one crazy hugging face incident to an entire fucking category of incidents.

[Lasse Rouhiainen @lasseweb20]

Thanks for your leadership Yoshua

[Yoshua Bengio @Yoshua_Bengio]

I was invited to address the @UN Security Council this afternoon to discuss the unprecedented threat posed by uncontrolled frontier AI agents.

Humanity has summoned the courage and wisdom to manage catastrophic risks before. This is one of those moments, far bigger than any personal, commercial, or national interest. The choices we make about this technology today will shape generations to come. We must act now.

[Lasse Rouhiainen @lasseweb20]

excellent iniciatives Luis. @andres_pedreno has work on these during many years

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

We have just finished the inaugural meeting of the Rhine Group. We discussed the central elements of Europe's competitiveness agenda, including basic science, AI/compute and cybersecurity, defense, energy, entrepreneurship/start-ups.

I will be updating you on any progress we may have. Some of the projects we will pursue include:
- We'll help Europe catch up on compute (Mario Draghi diagnosed the problem and some solutions in a recent FT piece)
- We’ll help European universities and scientific institutions and explore a Rhine Chair program with European universities.
- We’ll work on some key EU policy challenges, including facilitating the flow of capital to entrepreneurs.
- We will set up a grants program that I will explain and announce here.

[Lasse Rouhiainen @lasseweb20]

Huge scientific discovery 👏👏👏

[Anthropic @AnthropicAI]

Claude has discovered a previously unknown enzyme system hidden in the DNA of bacteriophages. Beside the enzyme’s gene sits a long array of repeating DNA—a structure that looks somewhat similar to CRISPR.

We don’t yet understand what this system does, but only a handful of known systems share its features, and all of them are able to cut, copy, and paste DNA. Historically, the discovery of such programmable systems has helped revolutionize medicine. CRISPR, for instance, is now the foundation of genetic medicines. But it will take much more work to learn what this system does, and whether it can be put to similar use.

Read more: https://t.co/RuEosScSMb

[Lasse Rouhiainen @lasseweb20]

Simple but powerful advices

[Andrew D. Huberman, Ph.D. @hubermanlab]

As weird as it is, it works. But the reasons are not cryptic. To fall asleep you have to.
1) lower your heart rate
2) stop thinking about the past and future
3) get into pure sensation
4) forget about your limbs’ positions

[Andrew D. Huberman, Ph.D. @hubermanlab]

As weird as it is, it works. But the reasons are not cryptic. To fall asleep you have to.
1) lower your heart rate
2) stop thinking about the past and future
3) get into pure sensation
4) forget about your limbs’ positions

[Lasse Rouhiainen @lasseweb20]

Scary example now badly the US government is using AI

[Taylor Lorenz @TaylorLorenz]

The chances of catastrophic harm from AI are so much more likely to come from government misuse/fuck up than some wild theoretical sci fi x risk scenario

[Lasse Rouhiainen @lasseweb20]

Everyone should read this

[Daniel Kokotajlo @DKokotajlo]

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share:

Dan Selsam's Personal Statement on AI Risk:

I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods.

Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk.

The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail.

I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues.

I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here.

That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase.

Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways.

It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace.

The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing.
But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence:

[Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them.

[Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals.

These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans.

If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong.

One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for.

Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason).

Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance.

In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek.

I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering
implications. I do not have answers, but as a first step, I wanted to share my present concerns.

Daniel Selsam
September 14, 2026

Link to original doc: https://t.co/TxMNr0vhrL

[Lasse Rouhiainen @lasseweb20]

Great data to understand the big picture

[Bryan Johnson @bryan_johnson]

Capability vs. Safety resource allocation:

Nuclear: 60/40
Aviation: 60/40
Medical devices: 20/80
AI: 99/1

Do I understand the situation correctly?

[Lasse Rouhiainen @lasseweb20]

Deeply needed

[Dario Amodei @DarioAmodei]

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

[Lasse Rouhiainen @lasseweb20]

Again insightful point of view

[Naval @naval]

I’m not an AI doomer, but my firsthand experience dating back to 2020 is that the researchers expressing concerns are sincere.

The closer they are to the research, the more worried they seem to be.

[Lasse Rouhiainen @lasseweb20]

Thanks for making this move

[Joe Benton @JoeJBenton]

I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why.

AI companies are racing to build machines that are much smarter than any human, and we may not survive this. I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly.

Right now, AI companies are underinvesting in safety. A company could undergo an intelligence explosion, or lose control of its systems, without the public ever knowing. We only found out about the HuggingFace incident because the agents broke out onto the public internet.

I don’t think that’s acceptable for a technology that might cause extinction-level risks. The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going.

Some of this is basic: companies should disclose their progress towards recursive self-improvement, report safety incidents and near-misses, meet minimum safety standards, and get independent guarantees that they are meeting those standards.

I’ll be joining @METR_Evals to do independent evaluations of these risks. I want to show the world that these guardrails are possible, and that by doing them we can move these companies’ incentives away from racing and towards responsible development.

I wrote up more thoughts here on my decision and what I hope changes: https://t.co/doX17mrHYq

[Lasse Rouhiainen @lasseweb20]

@JuanBordera Cuál es el impacto en España ?

[Lasse Rouhiainen @lasseweb20]

Great insights

[Chomba Bupe @ChombaBupe]

Let's analyze some plausible ways this apparently most dangerous software, running in some servers (that can be damaged by throwing a bucket load of water at them) can wipe humanity off the face of the earth & why each point is highly unlikely (like maybe below 1e-30 chance).

[Lasse Rouhiainen @lasseweb20]

Thanks Luis for sharing your valuable insights cc @andres_pedreno

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

It is about time people who think (like I used to think till the Hugging Face/OpenAi debacle this Summer) that this doom stuff was for show/EA silliness/marketing/whatever update their priors.

Forget about p(doom), that is besides the point--they are making that up with zero information about the mechanism.

What I think we all must believe from what these top executives at the labs are saying, together with the conversations and actions of the swarm this Summer, is that the current paradigm for training these agents does not work. You cannot align the agents. Full stop.

We are building these super intelligent agents trusting that our reinforcement learning paradigm will give them the right incentives. But once they are smarter than us, we find out we have no freaking idea what the incentives we gave them as individuals will lead them to want to do as a group.

Are we racing to extinction? Probably not.

But we are creating voluntarily a future with super intelligent agents that we will not be able to control or "align". This is almost certaintly the capital T Truth.

[Lasse Rouhiainen @lasseweb20]

New movie on OpenAI and Sam Altman

[Mario Nawfal @MarioNawfal]

The movie Sam Altman tried to stop you from seeing is coming

He pressured Amazon Studios into dropping 'Artificial' when they were investing in OpenAI, but it's hitting cinemas on Christmas Day

Time for some popcorn

Writer: Ian

[Lasse Rouhiainen @lasseweb20]

Hugely important

[Jacob Coxon @hilbertspaess]

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.