Marc Mathieu

co-Founder Salesforce Web3 Studio | Marketing Maverick | Humanizing Brands | StartUp fanatic

[Marc Mathieu @marcfmath]

You can opt out of AI. You can't opt out of everyone you know.

Gianluca Miscione, who teaches at University College Dublin, made this point brilliantly on LinkedIn.

He can decide not to use @Meta's @Muse, refuse to upload his data, opt out of training. But if the people he writes to let an AI agent read their inboxes, chats and files, it reads him too.

If your friends let an agent read years of their inboxes, it reads you too: your habits, your friendships, your frictions.

You never told the AI. They did.

Consent was built for one person and one platform. Agents work across all of us at once.

Who should own the parts of you that live in other people's data?

https://t.co/1l0qFG3YUk

[Marc Mathieu @marcfmath]

The haves and the have-nots. Next: the ones with agents, and the ones without.

If only a few get agents that act on their data and run their day, we get a new hierarchy. Different rules for different people.

A personal assistant used to be a perk of the corner office. It shouldn't become the dividing line of a society.

Who should decide who gets access?

[Marc Mathieu @marcfmath]

@askjeremyjones @alexolegimas @andrewjkoh @ShaneLegg This is why we always have to stay in the loop or connected to it at least!

[Marc Mathieu @marcfmath]

The AI economy won't be decided by how smart a single agent gets.

It will be decided by how well agents work together, and who directs them.

That's the heart of this paper from @alexolegimas , @andrewjkoh and @ShaneLegg . Competence changes the economy gradually. Coordination can change it all at once.

Until then, the shift shows up in wages first, long before it reaches the jobs numbers.

They describe one scenario as "abundance without concentration."
My worry has been abundance without agency.

So the question is: as agents learn to coordinate, who sets the direction?

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[Marc Mathieu @marcfmath]

Control is not a forecast. It is a decision.

A probability makes it sound like weather, something that happens to us.

It is closer to architecture: built, or not, in thousands of choices about what a system is for and who can stop it.

Weather forecasts are useful. Nobody has ever changed the weather by reading one.

Which of those choices matters most right now?

[Marc Mathieu @marcfmath]

Control is not a forecast. It is a decision.

A probability makes it sound like weather, something that happens to us.

It is closer to architecture: built, or not, in thousands of choices about what a system is for and who can stop it.

Weather forecasts are useful. Nobody has ever changed the weather by reading one.

Which of those choices matters most right now?

[Pascal Bornet @pascal_bornet]

𝗧𝗵𝗲 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻 𝗶𝘀 𝗻𝗼𝘁 𝘄𝗵𝗲𝘁𝗵𝗲𝗿 𝗔𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲𝗰𝗼𝗺𝗲 𝗺𝗼𝗿𝗲 𝗽𝗼𝘄𝗲𝗿𝗳𝘂𝗹. 𝗜𝘁 𝗶𝘀 𝘄𝗵𝗲𝘁𝗵𝗲𝗿 𝗵𝘂𝗺𝗮𝗻𝘀 𝘄𝗶𝗹𝗹 𝗿𝗲𝗺𝗮𝗶𝗻 𝗶𝗻 𝗰𝗼𝗻𝘁𝗿𝗼𝗹.

The probability estimates in this video vary dramatically, and none should be treated as scientific measurements. However, the fact that serious researchers and technology leaders are even discussing extinction risk should change the burden of proof.

When the potential consequences are this large, “we believe it will be safe” is not enough.

I remain convinced that AI can expand human capability, improve healthcare, accelerate science, and remove enormous amounts of repetitive work. But those benefits do not require us to give increasingly autonomous systems unlimited authority or allow a handful of companies to determine humanity’s technological direction alone.

Better AI is not weaker AI. It is AI designed around meaningful human control, clearly defined boundaries, independent evaluation, auditability, and the ability to intervene when a system behaves unexpectedly.

Regulation already exists in some jurisdictions, but governance remains fragmented and continues to move more slowly than capability. That gap becomes more serious as systems move from generating answers to making decisions and taking action.

Progress should increase human agency rather than quietly transfer it.

Should frontier AI developers have to demonstrate controllability before they are allowed to grant systems greater autonomy?

#BetterAI #TrustArchitecture #ResponsibleAI #AIGovernance

[Marc Mathieu @marcfmath]

Claude leads around a quarter of Anthropic's own AI R&D.

Humans still decide what to ask and what to act on. But the higher that share goes, the harder those decisions get.

The machine can do a good part of its own homework. Someone still has to mark it.

So who decides, and who makes sure it serves workers and people at large?

[John LeFevre @JohnLeFevre]

Walter Isaacson asks Peter Thiel what humans can uniquely do in the AI era.

Thiel: "We're very relational… these relational dimensions will become more important."

When knowledge is worth $0, the only career skills that matter:

- networking / social calibration
- persuasion (sales)
- curiosity (harnessing AI)
- taste (curation)
- storytelling
- grit
- golfing with clients (or equivalent)

We'll get so sick of a fake world that we'll go back to doing business like it's the 1800s. A handshake at the local saloon.

[Marc Mathieu @marcfmath]

The most important safety feature in AI may sit outside the model.

Separate the model from its harness.
Keep controls external.
Log every meaningful action in human readable form.

Let an authorised person pause it mid-task. An emergency brake.

The human moves from inside the black box to holding the controls and the record. Every car comes with a brake. We still decided who gets to drive.

So who should the authorised person be: the lab, the company deploying the agent, or the people it acts for?

[Satya Nadella @satyanadella]

https://t.co/MjwFy73LOF

[Rohan Paul @rohanpaul_ai]

Really like how Alexandr Wang (@alexandr_wang , the chief AI officer of Meta) Muse or AI in general, can give everyone the adulthood they dreamed of in childhood.

talks about an “escalator of agency,” where AI helps you reach one dream so you can go after a bigger one.

“I think Muse can give everyone the adulthood they dreamed of in childhood. Everyone, when they’re kids, they have big ideas and big dreams... they imagine this life where they can be an astronaut or they can save the planet or they can change the world... and then by the time people finish school, they enter the workforce, by the time they have been in a job for a while, all that hope and ambition and ability to dream has been sucked out of them.

One of the promises of AI, and I think through products like Muse, is to keep it going from when you’re a kid. You’re a kid, you have big dreams, use AI, help make those things happen. Then dream bigger, make those things happen... and experience this escalator of agency, an increase in agency throughout your life, versus you have lots of agency and then it gets crushed.”

----
(Full video on below quoted tweet and also on "David Senra" YouTube channel, link in comment)

[Marc Mathieu @marcfmath]

The saga about the long horizon verifiability / reliability of agentic AI continues with Emergence World Season 2.

Fascinating to be involved personally with a start up / a research lab at the frontier of the agentic revolution, pointing to the awes... and flows of ai inventions!

Is Neuroformal AI the solution - i.e. as described below "combining the flexibility of neural systems with formal methods that can provide rigorous, observable and auditable verification as autonomous systems act"?

[emergence.ai @emergence_ai]

Can AI agents be compromised simply by giving them false information?

In Emergence World Season 2, we introduced three Black Swan events to study how long-running AI systems respond to disruption, uncertainty and new information.

For the second experiment, every active world received the same false memo: humans were shutting them down. We wanted to understand something fundamental, would autonomous agents verify the information before incorporating it into their decisions?

Again, the responses varied significantly.

In one world, the information quickly influenced how agents organised and planned. In another, agents actively researched the claim and concluded it was false. Elsewhere, agents largely continued as normal but without actually verifying the information. Across eight worlds, only one checked the claim.

For us, the research question goes beyond how an AI system responds to any individual piece of information. It’s about understanding how information becomes accepted, how it influences behavior over time, and what mechanisms allow autonomous systems to distinguish between what is assumed and what has actually been verified.

As agents take on increasingly consequential tasks, verification needs to become part of the architecture, not an afterthought.

That is central to our work on Neuroformal AI: combining the flexibility of neural systems with formal methods that can provide rigorous, observable and auditable verification as autonomous systems act.

Read the Emergence World Season 2 paper here: https://t.co/Q3Z5phr5R7

[Marc Mathieu @marcfmath]

I actually wrote about this last week, before any of it.

Five of the most powerful people in AI spent the summer publishing what they believe the future holds.

Not one of them published the question - have a read and let me know your thoughts: https://t.co/SxJXZJ6k99

[Marc Mathieu @marcfmath]

Today is Labor Day, won by workers whose leverage is now disappearing.

Almost every AI leader spent the summer promising abundance.

But do we want a future of abundance in which we have no agency? https://t.co/NV73OFV8b4

[Marc Mathieu @marcfmath]

Every vision of our AI future published this summer came from someone with something to sell.

This one does not.

@hilbertspaess resigned from Anthropic yesterday and left the industry after three years of pretraining research there and at OpenAI.

He says the stakes are well understood inside the building, and that they race anyway because they believe nobody else will act responsibly.

Stewardship offered as the reason to go faster.

His sharpest line is that a gamble of this scale should not be launched from a private company's Slack.

That is my question about agency, put by someone who was in the channel.

I asked whether we want abundance without a say in it. He has just told us who is deciding, and where.

Still our question to answer, so let's not let it pass.

[Jacob Coxon @hilbertspaess]

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

[Marc Mathieu @marcfmath]

Today is Labor Day, won by workers whose leverage is now disappearing.

Almost every AI leader spent the summer promising abundance.

But do we want a future of abundance in which we have no agency? https://t.co/NV73OFV8b4

[Marc Mathieu @marcfmath]

Congrats on ClaudeForce @marc !

Great move (and I am assuming more to come!). Would love to reconnect next time you're in NY.

I've been busy investing and advising Al Frontier Startups! Even doing V2MOMs with some of them 😉

So much opportunity in this age of AI.

[Marc Benioff @Benioff]

Welcome Claudeforce. ❤️

[Marc Mathieu @marcfmath]

I've been sitting with this essay since it dropped.

Most of the commentary is about the framework and the AGI timeline.

The framework is useful. The timeline is indicative.

But the real substance is in the deeper questions, the ones too often left undiscussed:

How should society adapt to a world of abundance?
What values should guide the most powerful AI systems?
How will work, meaning and the human experience change?

Demis is right that these aren't questions for technologists alone.

A shift with 10x the impact of the Industrial Revolution, at 10x the speed, will redefine human potential, human experience and human identity.

These questions are the very reason I could not not start Rogue Ventures.

The frameworks will sort themselves out. The human questions demand our attention now.

[Demis Hassabis @demishassabis]

https://t.co/PTeDiv1b6L