By every measure, Opus 4.8 beats human experts as peer reviewers in a study of 124 Harvard social science papers covering a range from statistical analyses to formal models to qualitative work.
As academics, we need to figure out how to adapt the practice of science to AI, fast.
AI is an apocalypse in the sense that it reveals all. Or it reveals much.
We have mathematicians arguing that solving the problems that they themselves identified as the goals (in their funding applications and talks) never mattered.
We have the finest literary people in France who were collectively fooled by a novel written by AI. None of them would ever accept AI authorship.
We have been told for decades that without copyright, there could be no progress... and all of a sudden, we have these machines producing copyright-free content, often breaking new ground.
We have entire institutions that were supposed to be focused on innovation... that cannot adapt meaningfully to a world with AI.
We are left wondering. What is actual progress and what is mere signalling. How people react to AI tells us much.
"Now seven times three is twenty-one."
"And how do you know that?" asked the congressman.
"I just remember it. It's always twenty-one on the computer. I've checked it any number of times."
"That doesn't mean it always will be, though, does it?" said the congressman.
"Maybe not," stammered Aub. "I'm not a mathematician. But I always get the right answers, you see."
Isaac Asimov, The feeling of power
Sylvia Serfati sur la révolution de l'IA en mathématiques
"La plupart des résultats d'OpenAI mériteraient la médaille Fields".
"Les preuves sont d'une complexité incroyable et aussi parfois d'une beauté impressionnante. C'est-à-dire que nous n'avons pas le monopole de la beauté."
@AndrewCurran_@goodside “You did this from your house?”
“You don't hack a bank across state lines from your house!”
“It's stupid, man. It's universally stupid.”
@xavier_delmas Certainement cela semble dangereux car l'IA nous rendra dépendants.
Tout comme l'agriculture, la
médecine, l'électricité, la technologie..., bref la civilisation 🙃
@cremieuxrecueil yes but it's another system with low compute footprint.
and well, if it automatically launched predicted prompt to improve current analysis, you would never get your answer (and still pay for tokens).
so the equilibrium is *your arbitrage how far you want to go.
@cremieuxrecueil What would you say if someone did it to literally every problem you were working on? And if it was a machine?
Do you think it may happen soon? Is collecting data a moat? These are really interesting questions imho.
@__paleologo Probably of Polish descent, yes, but almost surely he himself as everyone in his neighborhood pronounced it ʃɔ-leh-SKI (the Polish original Cholewski would start with 'h' as in 'hollow').
Let's say a frontier model is a 10T MoE on GB300 NVL72. That's ~5 Joules/token or ~50 W at human thinking speed vs our brain at 20 W. The energy grid is $0.08/kWh and eating rice $0.50/kWh. So on an energy-per-thought standpoint rn we're surprisingly similar to frontier models!
I haven't had any issues with the rise of highly capable models, which go far beyond my own capacity to understand. It's perhaps much easier to adapt to such a perspective when you are an average mathematician and you've always felt part of a community, rather than being the most celebrated individual.
Over the past year, I've felt more productive than ever and embraced the possibility of connecting the dots with others, with models, with whatever seems necessary.
Be part of the hive mind, rather than pretending to be a self-proclaimed mastermind. There is no mastermind. We are a group of humans empowered by civilization, which is itself an accumulation of knowledge from thousands of generations of individual humans.
Picture: a screenshot from The Cathedral, a short film by @Ba66ins, based on a story written by Jacek Dukaj.