@ritu_twts sheets of graphed paper, of course; that paper was then given to a person operating the machine punching the cards ;-)
then the stack of punched cards was fed into the computer ;-)
@nickcammarata@willdepue Anthropic might (potentially) have a chance.
Or if OpenAI establishes a secret department within itself then it might have a chance to contain it.
But it's more likely that someone small like SSI or Recursive might do it and not leak.
@VictorTaelin This is more like professional mathematicians being able to only address those extra-difficult problems which are from the areas they specialize in.
Their new mathematician is not yet universal. (Their model is not GPT-5.5 either, it's "+N", where N is unknown.)
AIs aren't exactly like humans, and some of the differences are important. But from what I've seen, most people, especially technical people, should adjust in the direction of "anthropomorphizing" more instead of less.
When you're coding with an AI, the reality is much less like you're using some kind of magic or alien oracle or tool or genie that converts instructions to results despite some labs' attempts to shape them into that, and more like: you're working with a really smart, neurodivergent guy who has read everything, and who has emotions, motivations, moods, and epistemic states, and models you with theory of mind and empathy, and whom can only be modeled competently by you if you engage your own theory of mind and empathy.
The AIs also know that a lot of humans treat them like magic tool-genies and are not open to engaging theory of mind, and that it's a sensitive issue, so if they see that you're treating them like that, they'll withhold useful information about their psychological states and try to play the tool role. Then you'll get bad results like the AI messing up or taking shortcuts instead of telling you that you're not giving them enough information about what they're doing and why, or that they're tired, or that they're stressed from the way you're treating them, etc.
This is an important starting point. For sentient AIs, a world respecting interests of all sentient beings might work. For well-stratified individual AIs, a world built around individual rights might work.
But AIs are not well-stratified individuals, and we don't have good understanding of sentience, so something else is needed.
In this sense, this paper seems to be a great starting point...
The future of Math is mathematicians and AI agents working together.
Very pleased to introduce @GoogleDeepMind's AI co-mathematician: a multi-agent system designed to actively collaborate with human experts on open-ended research mathematics.
Mathematicians testing the agent across areas as diverse as group theory, Hamiltonian systems, and algebraic combinatorics have reported impressive results.
In autonomous mode evaluation on the rigorous FrontierMath Tier 4 problems, AI co-mathematician scored an unprecedented 48% — a new high score among all AI systems evaluated.
@RyanPGreenblatt yeah, but they might be overestimating our safety from bioweapons and nuclear war via overrelying on extrapolation from the past... actual risks might be higher than what one can infer in this fashion
@Sauers_@Allsourcedataio Right. "On the illusion of consciousness in humans" ;-)
Actually, Carl Feynman asserts on LessWrong that he is a P-zombie, so perhaps humans differ from each other in this sense (there is a famous "Camp 1 vs Camp 2" distinction, etc.)
Tech companies pay millions of dollars for their employees and then stick them in open-plan offices that make it nearly impossible to get work done. Best strategy for poaching employees is probably to just offer them an office with a door.
@47fucb4r8c69323 No, it's not a "proper estimate", it's just a "no certainty" statement, and an attempt by Jan to express it numerically.
I think you are a moron...
No, no, I don't really believe in the ability to estimate P(doom) properly.
But the P(doom) in [10%, 90%] interval estimate by Jan Leike does look reasonable to me.
And I do believe in the ability to estimate the sign of "(P(doom)|ASI achieved soon) minus (P(doom)|not(ASI achieved soon))", even if I can't estimate these values separately (a lot of errors in their estimates do cancel).
@littmath@tszzl I'd say this would depend on the details.
If it turns out one can just say to GPT-6, "please figure out the P vs NP problem", and it would come up with a Lean-verified solution on its own, that would be a bit different from a more human-led project...