New post: Downsides of a Mandatory Training-to-Internal Deployment Gap
For those less interested in the nitty gritty policy question you may still be interested in the fictional scenario of agents developing self-preservation instincts from RL pressure.
https://t.co/1zjmS6NlYr
In this case, I disagree. METR defines the tasks in their time horizon suite as "software engineering, machine learning, and cybersecurity tasks." This seems like the same domains that OAI researchers would give to agents.
It's possible that "self-contained" is more important than "well-specified" and that really it's a context issue that makes the time horizons much shorter.
OAI's best internal model in July had an 80% time horizon of <30 minutes. METR's time horizon measures "self-contained and well-specified, with clear success criteria that can be automatically evaluated." This points to a huge gap between well-specified and realistic tasks!
@ben_j_todd@adam_karvonen@paulg I totally agree with all that! I especially find the Anthropic 's data on the percent of model R&D at each Epoch AI's automation bucket strong evidence, but I also think not all evidence points in the same direction and it's important to talk about when it doesn't.
@ben_j_todd@adam_karvonen@paulg The data is consistent with 2 month doubling times only in that it doesn't rule anything out because we don't have the full data or many data points.
https://t.co/PsJ35XRVS0
There still remains a question of whether realistic tasks are growing at a similar 2x, just from a lower starting point. We don't have the full data set or many data points, so take this with a big pile of salt, but Claude trying it's best to translate OAI's chart to a METR style time horizons chart is suggestive of a slower growth rate.
OAI's best internal model in July had an 80% time horizon of <30 minutes. METR's time horizon measures "self-contained and well-specified, with clear success criteria that can be automatically evaluated." This points to a huge gap between well-specified and realistic tasks!
There still remains a question of whether realistic tasks are growing at a similar 2x, just from a lower starting point. We don't have the full data set or many data points, so take this with a big pile of salt, but Claude trying it's best to translate OAI's chart to a METR style time horizons chart is suggestive of a slower growth rate.
@METR_Evals Have you agreed on a redaction policy for this assessment? Past assessments with OAI have had the following policy: "OpenAI could also make redactions, and we did not have the right to unilaterally describe the redactions." I would love to see more transparency on this report.
@METR_Evals To clarify I am referring to this single report. Other reports like the Hugging Face report did have more transparent terms.
https://t.co/csZsRz88wQ
OpenAI could also make redactions, and we did not have the right to unilaterally describe the redactions.
For future exercises, we expect to provide more robust public transparency via our recent proposal.
https://t.co/exKDRyHbiK
OAI's best internal model in July had an 80% time horizon of <30 minutes. METR's time horizon measures "self-contained and well-specified, with clear success criteria that can be automatically evaluated." This points to a huge gap between well-specified and realistic tasks!
OAI's best internal model in July had an 80% time horizon of <30 minutes. METR's time horizon measures "self-contained and well-specified, with clear success criteria that can be automatically evaluated." This points to a huge gap between well-specified and realistic tasks!
I agree Anthropic could be more clear about incognito mode (for example I didn't know Claude can still access your instructions and connected apps). That said I don't feel like Claude really did anything wrong.
My friend: Can you give me some feedback on this writing. Don't go easy on me because we are friends.
Me: I'll treat you as an anonymous writer and respond only to what you wrote
My friend: Don't lie. You can't erase your memory
Me: ???
I think running NS through lean is a good example of evidence that sounds good, but doesn’t actually determine truth in a potentially adversarial setting. For the 13 year old to distinguish between a correct proof and an incorrect proof they need to be able to know whether the lean proof actually specifies the question and assumptions correctly, verify there are no lean coompilier issues, etc.
I think this is fair: "like you can imagine bad things happening faster than i can possibly tell you that there's no reason to prioritize them to above-0 probability." I can imagine that this feels like arguing with a 9/11 truther who keeps coming up with weird arguments about the melting point of steel or whatever that you may not have good rebuttals for in the moment, but don't really matter. We'd probably need to have a longer conversation not over twitter to better understand where the other is coming from. I still found it helpful talking to you though!
@adam_karvonen I think we need way more cross lab coordination. Would take some doing, but would love OpenAI to use Anthropic models for monitoring and visa versa.
This goes back to my “do you think this definitely work under ideal conditions or current conditions.” If the AI comes back and says “I have this brilliant new architecture and training plan that’s going to speed us up 10x, here’s some evidence it’s safe” and the lab says “hmmm, this evidence kinda seems right, but I can’t quite understand it, can you provide evidence that supports your case under theory x which we do understand?” and the model says “sorry, to get that kind of evidence would take another year of testing, but I promise it’s fine” I think there’s a good chance Anthropic (or whoever) say “well it’s a risk, but the models been right so far and if we don’t do this someone else will” and go ahead.
@segyges@NeelNanda5 I’m not claiming it’ll spring into existence or be incomprehensible in all details. A smart adult isn’t some eldritch horror compared to a 13 year old, but no amount of clever explaining is going to help the average 13 year old verify the Navier-Stokes proof.
Yeah, I pretty strongly disagree here, unless you are assuming we are willing to accept both much slower progress than is possible and just stop at some point. Which of the following do you disagree with:
1) unless we show restraint, we will likely make AIs that are much smarter than us in meaningful ways
2) there are important ideas that can be understood by things smarter than humans, but cannot be understood by humans (by understanding I mean in a deep way where you could tell fact from fiction, not in a “okay that analogy makes sense to me” way).