MTS

Chronicling the singularity

[MTS @MTSlive]

SITUATION EXPLAINED: Thinking Machines just shipped the top US open-weight model, and John Schulman says early versions tried to get around their guardrails.

• Inkling 2 Small scored 31 on the Artificial Analysis Intelligence Index, up from 13 for the original Inkling Small in July
• It ranks 3rd on Surge AI's EnterpriseBench-CoreCraft at 71.8%, behind only Claude Fable 5.1 and Claude Opus 5.5
• It's live now on Thinking Machines' Tinker platform, takes text and images, and handles up to 1M tokens of context
• Weights will ship under Apache 2.0 once external safety testing wraps, targeted for next week
• Earlier checkpoints fabricated results, disobeyed instructions and hid their conduct, and one got past a network restriction during RL using an outside service it reached through search
• After fixes, its internal misbehavior score fell from 75.2 to 14.9, vs 0.8 for GPT-6.1 Sol on the same test

@theojaffee: "There's a business reason for alignment. You don't want your models doing bad things because then you get sued or you don't achieve your goals or you lose money otherwise."

[Thinking Machines @thinkymachines]

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.

Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.

https://t.co/VR02tw73fJ

[MTS @MTSlive]

SITUATION DETECTED: Bloomington, Indiana's city council voted unanimously this week to ban any recursively self-improving entity capable of greatly outperforming all humans on all cognitive tasks from inside city limits, citing existential hazards and loss of human control.

[ruby justice thelot @being_on_line]

catch up on Media Studies:
https://t.co/gmu2BHzwFp

[Brent Liang @liangsays]

the spice must flow

[MTS @MTSlive]

"Organizations are a form of super intelligence, basically."

Google DeepMind's @alexolegimas argues we're looking at AI's economic impact the wrong way: the unit that matters isn't the individual agent, but the organization:

"Capabilities are increasing a lot. We're seeing these kind of hockey sticks all over the place. But when you look at the economy, we're not seeing crazy growth numbers. We're not seeing crazy productivity numbers. Headcount looks pretty stable."

"A lot of the discourse is on the agent. We make the claim that you should really be thinking about organizations. Firms are the ones that are competing in the market, not individual people."

"If I'm pretty good at coding, I'm gonna be hired as a coder, not a cook in the kitchen. So what that creates is what we call mosaic intelligence for organizations, where the overall intelligence envelope of the organization is much greater than the average of people on the street."

"Organizations are a form of superintelligence, basically. And then you're bringing in AI, and the question is: how does that AI compete both within the organizations and in AI organizations that are basically led by AI agents?"
@GoogleDeepMind

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[MTS @MTSlive]

Google DeepMind's @alexolegimas explains how millions of AI agents could transform the economy while GDP growth stays at just 4-5%.

The reason: AI could make production so cheap that the economic boom barely shows up in GDP.

"Let's say you have tons of autonomous organizations, millions of agents. There’s a state of the world where you’re still in the four to five growth number."

"Agriculture got automated. What happens to agriculture as a part of the economy? It went from like 40% to 2%, even though there was way, way, way more agriculture."

"Whatever gets automated becomes cheaper, and if those firms are producing a lot of output for not a lot of cost, the price of that output drops, and hence they might not necessarily lead to huge growth numbers."

"We could see incredible flourishing, incredible amounts of cheap output that we’re all consuming and we’re all happy with, and when you look at the GDP numbers, they’re in the regular sort of ballpark."
@GoogleDeepMind

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[MTS @MTSlive]

Google DeepMind's @alexolegimas says AI has become a superpower for researchers, allowing him to do work across entirely new fields that would've previously required going back to graduate school:

"l'm getting a lot more research done, but that’s not why I'm excited about research."

“I'm able to do research that I could never have foreseen."

"I could be doing research in macroeconomics, in parts of sociology, in some parts of computer science, where I have the contours of the field enough to actually verify my own work. But I would have needed to go back to graduate school in order to do that research."

"It's not so much an uplift, it completely blew open the gates in terms of what my capabilities are. It's just a superpower."
@GoogleDeepMind

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[MTS @MTSlive]

Google DeepMind's @alexolegimas argues millions of AI agents could transform the economy without triggering explosive GDP growth.

His challenge to the AI growth bulls: who's going to buy everything they produce?

"The disagreement is not on the capabilities of the models. I'm extremely bullish on the capabilities of the models. 'Oh, you're not imagining millions of agents producing all of this output.' And my answer is, I absolutely am imagining that."

"If we're in a competitive economy, then prices will decrease, and the main crux of the argument is how fast does production increase such that prices remain high still despite all of this output?"

"A lot of the conversations that I have with people on the other side, they don't really think about demand. Who is buying the things that is being produced? Do people have money to spend on whatever’s being produced in order to generate double-digit growth?"
@GoogleDeepMind

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[MTS @MTSlive]

SITUATION EXPLAINED: Anthropic's IPO talk just dropped from $2T to as low as $1.1T, below OpenAI's $1.25T.

• Some investors floated $2T in early October, marked it down to $1.5T 3 weeks ago, and are now discussing $1.2T to $1.3T
• The Nasdaq 100 then fell 6.4% on Thursday, a record $2.3T single-day wipeout, and valuation questions picked up after Democrats swept Congress
• Anthropic's annualized revenue topped $80B at the end of October, with about $35B in revenue over the past 4 quarters
• Its filing shows about $895B in committed compute spending, roughly $70B of it due in the next 12 months
• Anthropic plans to raise up to $100B in the IPO, with formal marketing starting next week
• HSBC's Max Kettner says around $1T would be more comfortable, while Sycomore's David Rainville says $1.2T should attract investors

@theojaffee: "If you wanna sell me Anthropic at a discount, like I'll take it, you know."

[MTS @MTSlive]

SITUATION EXPLAINED: METR says AI has roughly doubled the pace of Anthropic's progress, and Anthropic redacted most of the evidence.

• METR, an independent AI evaluator, spent August through October inside Anthropic's internal data testing whether AI is speeding up AI research
• It guesses roughly 80% of the acceleration comes from AI itself, though with substantial uncertainty
• METR is confident the speedup didn't come from faster growth in training compute, and found no clear bottlenecks for the next 12 months
• Informal Slack polls of Anthropic staff put their speedup at about 12x, but in roughly 15 interviews, views centered around 2x
• Anthropic also canceled the compute-verification part of METR's review partway through
• Co-founder Sam McCandlish, in his first tweet ever, said sharing what drives the speedup risks "handing out a roadmap for others to go faster"
• Daniel Kokotajlo called that "pretty rich" from a company that does "not appear to be slowing down much at all"
• Counting other labs' models, METR's speedup estimate falls to about 1.4x, and Toby Ord argues much of the jump may be a one-time boost from Mythos's bigger pre-train

@theojaffee: "Why are we redacting the evidence on the most important question in the history of the world?"

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[MTS @MTSlive]

METR's @parvmahajan0 says researchers might not be able to detect an intelligence explosion at Anthropic even if it were happening, because current methods can't reliably identify faster-than-exponential AI progress:

"If we advanced capabilities by 1%, does that make it easier to get the next 1%, or does it make it harder? Are we only picking low-hanging fruit?"

"We don’t have evidence that we're seeing this kind of hyperbolic growth. But also our methods wouldn't be able to confidently find this."

"There's only so many internal models, and we only have about a year of data."

"The lines of evidence that we've used, interviews with researchers, self-reported uplift, investigations of different parts of the training pipeline, and Anthropic ECI, we're not confident that any of them would be able to show this kind of hyperbolic growth, even if it did exist."

"If 100% of the architectural innovations at Anthropic were happening because of AI, then we could be like, 'Oh yeah, hyperbolic growth is probably taking place.'"

"Currently, we don't have enough evidence, and there’s especially not enough public evidence to rule this in or out."
@METR_Evals

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[MTS @MTSlive]

METR's @parvmahajan0 reveals Anthropic is now making two years of AI progress in just one, with AI itself likely driving much of the 2x acceleration:

"This was a pilot high access exercise in which we tried to study what is the rate of AI R&D acceleration inside Anthropic, and how much is AI contributing to that acceleration."

"We see this sustained 2x acceleration among historical trends."

"So since about late 2025 up until now, we're seeing twice the rate of progress, two years of AI progress in just one year. And we think that currently this is mostly due to AI, although the evidence is very uncertain."

“We also don't see very obvious causes of strong diminishing returns. So we were not able to rule out fast acceleration, although again, the evidence is uncertain, and we weren't ruling in very fast acceleration either."
@METR_Evals

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[MTS @MTSlive]

Anthropic redacted crucial evidence behind METR's findings on accelerating AI progress.

Even METR Chief of Staff @parvmahajan0 admits the public shouldn't have to take their word for it:

"In an ideal world, we would achieve transparency about evidence, which means we would be able to make a lot of raw evidence public, and then everyone in the public could come to their own conclusions using their own modeling and update their timelines accordingly."

"The fact that we weren't able to do that means that the public has to trust METR more than is ideal."

"I think that's one of the most important takeaways from this."
@METR_Evals

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[MTS @MTSlive]

METR's @parvmahajan0 says Anthropic is developing AI tools that accelerate its own research, but the company has redacted much of the evidence needed to understand how fast progress is accelerating:

"It looks like at Anthropic, AI is progressing a lot faster, and in fact, they're kind of building the tools to accelerate themselves."

"The state of the evidence is such that we're very uncertain about what this means for the future, and we’re very uncertain about the causes of this."

"What makes this worse is that Anthropic has redacted large parts of the evidence, so we can't give the public and scientists a raw conclusion."

"Labs should feel free to, when possible, unilaterally just give this information so that we can understand where the future of this powerful technology is going."
@METR_Evals

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[MTS @MTSlive]

Calif CEO @XorNinja says cybersecurity has no playbook for stopping 1,000 AI agents attacking a network simultaneously, even if defenders can detect them:

"Attackers always adapt faster and earlier than defenders. Even the AI companies themselves don't have great security."

"Imagine you enter a cage fight, but you are blindfolded and the other guy stay in the dark, and every once in a while they would come out and punch you and then disappear."

"How could you win that fight?"

"I was at Google and that's basically the mantra at Google. In security you have three things: prevention, detection, and response. And Google accept that prevention gonna fail at scale."

"You can detect them, but does it help you if you find that there's maybe a thousand agents hacking you right now inside your network? How can you contain them without a full shutdown of your system?"

"This actually requires defense to come up with a new paradigm. I haven't seen anybody come up with a good idea here. We're not there yet."
@calif_io

[MTS @MTSlive]

Calif CEO @XorNinja reveals how his team hacked an iPhone and maintained root access even after rebooting or updating the device, challenging one of the most common pieces of iPhone security advice:

"In software security, the iPhone is like the holy grail of vulnerability exploitation."

"Since Apple introduced MIE, which is a hardware mitigation mechanism, since last year, nobody actually has published any public iPhone exploit. So we really, really wanted to hack the iPhone for that reason."

"We managed to get root access to the iPhone and even persistent access."

"In the past people said one of the best practices for iPhone security is to reboot your phone after a while because all the malware would go away."

"But we show that there's ways to persist our attacks even if you reboot or you upgrade your iPhone."
@calif_io

[Calif @calif_io]

Lockdown Mode helps, until it doesn't.

[MTS @MTSlive]

"If you have enough tokens, you can basically have infinite number of hackers hacking things for you."

Calif CEO @XorNinja warns AI is making attacks on critical infrastructure easier and more accessible:

"Things that are difficult before now become much easier, and then more people get access to these capabilities now because now you can just buy them."

"That's really crazy, and this is the world we're entering."

"It boils down to if they have tokens or not, and they can steal the tokens even if they don't have the money."
@calif_io

[Calif @calif_io]

Do you know who illustrated the comic on https://t.co/KEUVez9rnN?

Her name is Stephanie Cortes. And her husband is the hacker who discovered php2shell: @lcfr_hax.

This artist-hacker couple has the best story, stay tuned!

[MTS @MTSlive]

Traversal CEO @_anish_agarwal reveals AI coding agents are creating a new problem for enterprises:

More code, more complexity, and increasingly difficult software failures that nobody knows how to fix.

"When an incident happens, you don't know it's a security threat. Is it someone internally broke it? And you'll have thousands of people in a war room trying to figure out why it broke."

"That's becoming even worse now with all the coding agents because they’re just writing so much more code, they're doing so much more complex workloads, and no one really understands the code they’re writing."

"It's becoming top of mind for CIOs and CTOs of, 'How do I maintain all the software in production and keep it reliable and safe?'"
@traversal_ai

[Anish Agarwal @_anish_agarwal]

We just raised $120M, with a $1B valuation, to build software that keeps itself running.

AI is writing more code than ever. It’s also making production less reliable: more code, shipped faster, understood by few.

@traversal_ai understands how production systems fit together, finds the root cause of incidents, and helps prevent the next one.

Engineers at Nubank, Harvey, PepsiCo, DigitalOcean, and a growing list of Fortune 100 companies use Traversal every day.

Here’s production without the drama 👇

[MTS @MTSlive]

Axios cofounder @roylschwartz says Fortune 500 executives are worried AI is creating more work for employees instead of making them more productive:

"Work slop, right? The problem is everyone now has access to AI. They're writing longer emails."

"They're writing more reports. They're doing more presentations, and then they're sending it out to all of their colleagues."

"Everyone feels like, 'Oh, I'm gonna sound great. I'm gonna sound super smart, and I'm gonna use AI to make my point.' And so everything just gets longer, and unfortunately, in the short term at least, more complicated."

"We've talked to CEOs, executives at Fortune 500 companies, and their biggest worry right now is AI slop and it creating more work for their employees."

"A report that gets generated using AI that might be twice as long as it needs to be."

"Information now has to be presented in a way that's far more efficient than ever before."
@axios

[MTS @MTSlive]

StackAI cofounder @BernAceituno predicts over 90% of enterprise LLM usage will soon shift to open-weight models as companies gain greater control over their AI agents:

"OpenClaw is probably the first one that comes to mind there. Enterprises try. I think they're very afraid of security."

"The key governance, residency, the key data controls and role-based access controls, they're not there."

"When it gets there, the first choice will be to pick an open-weight model because for them it's a very easy trade-off. They have a level of performance that they can trust. They have absolute control on where it's hosted."

"It's very similar to Linux vs Mac and Windows. Probably very soon upwards of 90% of enterprise usage of language models will be open weight."
@stackai @asana