Neural networks might speak English, but they think in shapes.
Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.
Starting today, we’re releasing a series of posts on this research agenda. 🧵
Applications are open for Reconstellation 2.0!
Reconstellation is 90-day program where you can reignite the fire of your humanity by pursuing a project of your choosing within a community of many supporting elements, including:
• programs
• workshops
• coaching
• Established Creator talks,
• many forms of feedback
• and much more
Apply by Sept 18!
Day 1 is Sept 21
https://t.co/udIssbbYUm
I think it's important for OpenAI (and other frontier labs) to clarify and address these concerns in a considerate and thoughtful way, especially given how much value they are deriving from these announcements, and more importantly, the value they have derived from the work of the mathematical community, either through training data or otherwise.
Too dark.
The AI companies must drop the doom narrative and remain what they are, helpful software. Pay for service tools.
They should see that we don’t want them to solve the problems *for us*. They don’t even need to do that to make their riches.
Instead, they pick the lowest hanging fruit to deceive us that from here, everything else will be automatically solved. They pick problems that are amenable to massively paralleled computing and search in spaces like math where the rules are clear and defined, and then falsely extrapolate that everything is just that, a search problem. It is not true. It is a dangerous lie.
In this way they will kill progress by creating an incredibly hostile and secretive culture where no one wants to share anything as we expect everything to be scooped by them.
The work to be done is to *help people solve the problems*. Academia has been broken for a long time. The literature is a muddled mess of contradictions and unreproducible results. No one wants to do the work to ‘clean it up.’ No one knows what to fund because the foundation itself is shaky. Rebuild the foundation.
OpenAI and Anthropic could do this *for us*. They could use agents to scan the literature in spaces where the uncertainty is highest and surface core contradictions to be tested, run automated experiments to figure out ground truths, and share new work that reestablishes clarity so we can then build new discovery on top. A clean knowledge base; a springboard of sorts for collaboration and prosperity. This is work that no human alone could do. It is a grand project.
Our AI leaders have lost their way. Look back to what inspired humans for millennia, wonder and awe at the universe, at beauty, pride for our culture, a shared sense that we can build the future together. We do not want idiocracy or fully automated space communism. We do not want you to take 1000s of years of our artistic work, our souls, and reproduce it.
We want to have a part in the future. We would once band together to defend our way of life against a common enemy. We have now made that enemy ourselves. If we stopped to take a breath, to see the forest for the trees, we’d realise that this is the best way of life man has ever known, and must be defended.
By stepping back from the superintelligence narrative to one of a defence of our prosperous way of life, the AI companies can make their money simply by charging for the services they provide and and in return, we will continue to feed them our out-of-distribution ideas, wants, and wonder.
It does not need to be this way.
I hear roughly 3 reasons people keep working at AI labs despite believing in ~10% extinction risk:
1) Techno-determinism: Someone will build ASI no matter what, and I can do it better & more safely than China/OpenAI/etc
2) Consequentialism: ASI might kill us, but it also might produce utopia/immortality/superabundance, so it's a +EV bet
3) Self-interest: I am personally having fun & getting rich working on cool tech with friends. I don't think about the macro stuff.
Notably, none of this is "I'm hyping up the risk for marketing reasons." People believe what they say, while being capable of a lot of internal dissonance / compartmentalization / self-justification.
(Personally, I think the public is better served by AI researchers talking about & creating consensus for the specific safety solutions — regulatory or otherwise — they want vs. vagueposting about extinction.)
Btw, this is the only mathematician who has solved one of the seven Millennium Prize Problems.
he rejected both the Fields Medal and the $1M prize:
> be Grigori Perelman
> Russian mathematician
> 16 years old
> go to International Math Olympiad
> score 42/42
> years later…
> there’s a geometry problem called the Poincaré conjecture
> been unsolved since 1904
> generations of mathematicians fail to solve it
> Clay Institute puts a $1M prize on it
> wrestle with the problem for years
> 2002
> casually upload the solution to arXiv
> don’t even care about submitting it to a journal
> mathematicians spend years checking the solution
“fuck
he actually solved it”
> 2006
> offered the Fields Medal
> most prestigious prize in math
> refuse it
> 2010
> offered the $1,000,000 Millennium Prize
> refuse that too
“EVERYBODY UNDERSTOOD THAT IF THE PROOF IS CORRECT THEN NO OTHER RECOGNITION IS NEEDED”
not particularly important but "vortex" is a really good word. just really top tier. rolls right off the tongue. futuristic. not many words in math as good as vortex
Synthetic data derived from production user data of consumer AI tools is used for training. I’ve heard this rumour from both large labs’ employees.
In particular, if you’re doing something “interesting” like working on complex math/business/software/bio problems you’re dramatically more likely to get trained on because they filter/up-weight towards those usecases where the model has the most to learn.
Even in ZDR and “we won’t train on you” regimes, derivative data is usually carved out. The promise is only not to train on exactly the data you put in, rewritten data is fair game.
"While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models." oh fun times fun times
OpenAI has solved the Navier–Stokes Millennium Problem using an 'internal model that is significantly more capable than GPT‑6 Astra.' They began work on it after hearing rumors that the problem had been solved, and threw 10,000 concurrent agents at it. They solved it in 88 hours.