tom cunningham

Economics & AI @ @METR_Evals (ex-openai)

[tom cunningham @testingham]

@lugaricano FWIW I acknowledge it’s a weird situation where you know the rate of acceleration but you don't know the location or velocity. Like sitting in a car blindfolded , you can feel the g’s but you don't know how fast you're going or where you are.

[tom cunningham @testingham]

Hi Luis:

- concretely, the IRT is unique up to an affine transform, which allows us to make judgments of acceleration, though we cannot meaningfully talk about a level or proportional changes.
- the assumptions that get us this is that there exists a latent capability, & each of these benchmarks is a logistic function of that latent capability. Although we cannot individually test the assumptions they're over-identified, & in the report you can see we do tests for robustness (many other people have gone deeper on this, can share links).
- I think the best intuition is that we're doing a principal components analysis.
- there are of course many ways this could go wrong, we try to cover some of them in the report, & there's a mini litereature that's sprung up around interpreting ECI. It's a deeply unsatisfying metric but it's also the best we have.
- here's a visualization which I think is useful for building intuition: https://t.co/k6YrYDU3NZ
- the *quantity* that we're recovering is something that's slightly difficult to describe: it's latent ability to do benchmark tasks. As we discuss in the report, we are unsure how smoothly this is related to real-world impact (which itself is very hard to operationalize).

[tom cunningham @testingham]

@lugaricano great -- happy to talk more about your remaining doubts -- & maybe you want to post an update to your original post?

[tom cunningham @testingham]

@peterwildeford @TheDavidSJ @davidroodman @YafahEdelman @james_s48 @EpochAIResearch @AlexBarry4 email me! tom.cunningham @ metr

[tom cunningham @testingham]

@peterwildeford @TheDavidSJ @davidroodman @YafahEdelman @james_s48 @EpochAIResearch @AlexBarry4 I don't feel i've looked at ECI stuff to make confident calls, but this is useful background for the claims made in the Anthropic report, why we have plots of residuals there.

[tom cunningham @testingham]

@TheDavidSJ @davidroodman @YafahEdelman @peterwildeford @james_s48 @EpochAIResearch @AlexBarry4 https://t.co/6r4FgTWqyC

[tom cunningham @testingham]

@TheDavidSJ @davidroodman @YafahEdelman @peterwildeford @james_s48 @EpochAIResearch @AlexBarry4 can see residuals for ECI here:

[tom cunningham @testingham]

@TheDavidSJ @davidroodman @YafahEdelman @peterwildeford @james_s48 @EpochAIResearch @AlexBarry4 just to check (1) they're not actual benchmarks, these are synthetic? (2) did you do the calibration check on residuals that we do in the note?

[tom cunningham @testingham]

@datagenproc aristotle will recognize his true heirs

[tom cunningham @testingham]

When Aristotle is reincarnated he will not say his problem was lack of data. He will see that we're working with mostly the same old jigsaw pieces but we've put them together in a different way, & he'll recognize it's better.

The dolphin is next to the dog, not the fish. The far East is next to the far West. The sun belongs with the stars, the moon belongs with the earth. Heat is motion. The different races and sexes of humans are closer to each other than he had thought, and they're also all fitted in very close to the apes and the monkeys.

We'll just need to show the jigsaw to Aristotle, how we've rearranged all the pieces that he already had available, and he'd recognize that it's right. We wouldn't have to show him new facts, just new theory. We've accumulated a lot of new pieces but the patterns that those new pieces helped to form are, in retrospect, visible in the pieces he already had.

There are also collections of pieces which Aristotle was working on fitting together but which we have given up on. Early philosophers tried to slot together the different types of Virtue, the different types of Republic, the different types of Friendship, as if they came out of some periodic table -- the four humours, the seven vices. That expectation is much weaker today.

[tom cunningham @testingham]

@TheDavidSJ @davidroodman @YafahEdelman @peterwildeford @james_s48 @EpochAIResearch @AlexBarry4 FWIW I find this visualization most useful: https://t.co/AMKb9FPmxW

[tom cunningham @testingham]

@TheDavidSJ @davidroodman @YafahEdelman @peterwildeford @james_s48 @EpochAIResearch @AlexBarry4 what exactly are family A & B here?

It sounds like family B, they're not linear in log-odds, & the linearity-in-log-odds assumptions are jointly testable (& we discuss this in the report).

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[Vincent @vvvincent_c]

@testingham @slimshetty_ the humans stay winning

[tom cunningham @testingham]

https://t.co/FQXXZDOofV

[tom cunningham @testingham]

You can list to some 1965 lectures by Isaiah Berlin on "the Roots of Romanticism" as podcasts. Very good.

In a nutshell he's saying that in the late 18th century, after thousands of years, Western thought finally hit the ceiling of rational inquiry. The Romantic movement is just the sound of European intellectuals banging their heads against that ceiling.

He says that there came across Europe a general and explicit realization that the jigsaw has no solution; the mess cannot be untangled; the shards do not form a whole.

There is no rational reconstruction of religion, morality, aesthetics, justice, social order. There is no transcendent form which human things approximate. There is no higher truth about who is right or wrong. There is no deeper principle which can reconcile two conflicting imperatives.

The manifestation of this realization, Berlin says, is the Romantic mindset: a sudden drying up of the appetite for earnest inquiry. There is no buried treasure so put down your spade. A turn towards nature, art, irrationalism, authenticity, commitment to a cause (whatever the cause may be), primitive and ancient things, the sublime.

This also gives an interpretation of subsequent intellectual history. There were additional attempts to systematize and make sense of human ends (Utilitarianism, Marxism, Logical Positivism, Modernism, Welfare Economics, Rationalism), but they tend to be adopted only by the less clear-sighted, and attempts to make them intellectually rigorous fizzle out. The mainstream position has instead been either to avoid talking about fundamental ends at all or to adopt some kind of explicit irrationalism (Existentialism, Postmodernism, Pragmatism).

Isaiah Berlin himself is famous as an advocate of muddling-through: he says somewhere else that all the problems of the 20th century came from taking abstract ideas too seriously.

[tom cunningham @testingham]

What are the alternative interpretations? (1) That there is some truth of the matter, the Romantics gave up too early, they needed to enquire just a little farther. (2) That the discoveries of the 18th century were not new: there has always and everywhere been a tacit acknowledgement that philosophical and religious systems are fables, that they are plaster facades built on the front of brick houses.

The other thing that jumps out: the story is all European. I'm not sure what he'd say about thought in the other half-dozen continents, people who've been trying to untie the same knot.

[tom cunningham @testingham]

@sayhar here! https://t.co/FQXXZDOofV

[tom cunningham @testingham]

@peligrietzer (I would put virtue ethics halfway between the "not fully worked out" and "explicitly irrationalist" poles; and my guess is MacIntyre would endorse this)

[tom cunningham @testingham]

@peligrietzer My answers to your answers :) https://t.co/7SeujGieU9

[tom cunningham @testingham]

You can list to some 1965 lectures by Isaiah Berlin on "the Roots of Romanticism" as podcasts. Very good.

In a nutshell he's saying that in the late 18th century, after thousands of years, Western thought finally hit the ceiling of rational inquiry. The Romantic movement is just the sound of European intellectuals banging their heads against that ceiling.

He says that there came across Europe a general and explicit realization that the jigsaw has no solution; the mess cannot be untangled; the shards do not form a whole.

There is no rational reconstruction of religion, morality, aesthetics, justice, social order. There is no transcendent form which human things approximate. There is no higher truth about who is right or wrong. There is no deeper principle which can reconcile two conflicting imperatives.

The manifestation of this realization, Berlin says, is the Romantic mindset: a sudden drying up of the appetite for earnest inquiry. There is no buried treasure so put down your spade. A turn towards nature, art, irrationalism, authenticity, commitment to a cause (whatever the cause may be), primitive and ancient things, the sublime.

This also gives an interpretation of subsequent intellectual history. There were additional attempts to systematize and make sense of human ends (Utilitarianism, Marxism, Logical Positivism, Modernism, Welfare Economics, Rationalism), but they tend to be adopted only by the less clear-sighted, and attempts to make them intellectually rigorous fizzle out. The mainstream position has instead been either to avoid talking about fundamental ends at all or to adopt some kind of explicit irrationalism (Existentialism, Postmodernism, Pragmatism).

Isaiah Berlin himself is famous as an advocate of muddling-through: he says somewhere else that all the problems of the 20th century came from taking abstract ideas too seriously.