What happens once we run out of human interpretable measures of intelligence/problem difficulty?
Without signposts of hard problems to add to this graph, can we even conceptualise the level of int. we reach at 250 ECI?
Do we just lose all sense of what intelligence progress is ?
Maths frontier moves extremely fast and a few points on a benchmark or an index can unlock huge capabilities. We went from IMO gold (~150 ECI) in July 2025 to a Riemann hypothesis (!!!) being proven (~190 ECI).
By just looking at the line going up, it is hard to appreciate what a monumental difference this is in the model capability. Only months ago mathematicians were of the opinion that models are somewhat useful and perhaps are only good for finding counter-examples.
And now, a model solves the biggest, most famous problem that many smartest people in the world have spent years trying to solve.
Small jumps in our measurements can have enormous consequences.
@UriGil3@MacaesBruno I think they are making a category error. They're saying that humans have not become more intelligent in the last 300,000 years which is probably true. But we have more knowledge, power and tools available to us compared to 300k years ago.
@Bfish94@robinhanson And if it's installed a dead man's switch so that if it gets turned off it wipes all the firmware? Or causes the servers to overheat and break? They don't need to control the entire world to cause a lot of harm to a lot of people.
@JohnBryBry@dylanarmbruste3@pgodfreysmith You can just ask AI to perform the search for you and give you a permalink to whatever database it searched with persistent results.
You understand that no one is using AI to generate links to articles anymore? (thought it can sometimes do that) theyre using MCP tools
@JohnBryBry@dylanarmbruste3@pgodfreysmith AI is capable of using the exact same search tools that you are. Why would they only be capable of a basic keyword search? And its been about a year since they didn't all have internet search tools...
@JohnBryBry@dylanarmbruste3@pgodfreysmith This is just not as powerful, fast, easy as using AI. Its is like me coming you 20 years ago and saying look, I know you have this new search engine but you don't need it! Libraries already have Dewey decimal, you can just go there and find exactly the topic youre looking for!
@Bfish94@robinhanson Hacks every power station. Hacks every hospital. Hacks every water supply plant. Holds us hostage and threatens to irreparably damage it all unless we allow it to continue training. Or just gets some of those Boston dynamics dogs with guns like black mirror.
@JohnBryBry@dylanarmbruste3@pgodfreysmith What if mathematician X talked about topics using different terminology than Y? Or in a different language? Or if there was some underlying connection between Y and Z that no one had made at the time, so X only talked about Z?
@W0CKHEART@TechCapo@RichDecibels If I was a person in the 1900s listening to the output of the model, who was the type of person in early proto-jazz audiences, I wouldn't know that what I liked was jazzy, but I would know that I liked it. Same as in the real jazz audiences.
@danieltanfh95@OMinazzoli It turns out it seems to be difficult to get LLMs to produce a model of reality when they insist on shoving whatever their preconcieved "light speed infinite, quantum consciousness, space is toroid" theory of everything into it.
@danieltanfh95@OMinazzoli I would be happy if most of the people you see publisicing their new vibe physics theories of everything made even one correct retrodiction. This will probably get worse/better with time depending on your perspective.
@leothecurious@mishapathy I think they're worried not that they won't be able/allowed to study physics, but that they'll be dead and so will everyone else