Fortress Accident

Asylum for the feeling

[Fortress Accident @_postal_]

@deepfates Quality?

[Fortress Accident @_postal_]

@nosilverv sometimes I find old claude 3 threads and run opus 4.8 on top of them

[Forrest Cardamenis @FCardamenis]

Baby has gotten good at identifying “dada” but still struggling with surrealism, cubism, even baroque.

[Fortress Accident @_postal_]

@danshipper precisely the windows when you can make major life decisions, after that return to homeostasis

[Fortress Accident @_postal_]

@ben_j_todd two years? happening right now

[Fortress Accident @_postal_]

there’s more

[Fortress Accident @_postal_]

Deutsch, Chalmers, Pearce, Chapman… seems you have to be a David to play in the consciousness field

[Fortress Accident @_postal_]

3 AI futures
1. utopia, new discoveries, wide range of problems solved, unimaginable boons for everyone - free energy, new materials, diseases cured
2. shitty cyberpunk, AI in water supply manages everyone, 1% syphons all value from 99% permanent underclass
3. butlerian jihad

[prinz @deredleritt3r]

You don't truly understand the magnitude of the potential impact of powerful AI on the world unless you are aware, and have fully internalized, that senior leadership and most researchers at the frontier labs *actually believe* the following:

1. Existing AI is already significantly speeding up AI research. Very soon (this year), AI will very likely take over *ALL* aspects of AI research other than generation of novel research ideas. Soon (within the next 2 years), AI will very likely take over *ALL* aspects of AI research, period. This means hundreds of thousands of GPUs working 24/7 to discover novel ideas at the level of, or better than, the likes of Alec Radford, Ilya Sutskever, etc. The thread below presents a conservative timeline: AI researchers will "meaningfully contribute" to AI development in 1-3 years.

2. Many (but, as far as I can tell, not all) executives and researchers at the frontier labs believe that fully automated AI research will kick off recursive self-improvement (RSI), wherein the AI models will autonomously build better and better AI models, with human oversight (for safety reasons), but increasingly with no human input into the research or implementation of that research. From the thread below: "'[h]umans vs AI on intellectual work is likely to be like human runner vs a Porsche in a race', likely very soon" - but replace "intellectual work" generally with "AI research" specifically.

RSI is a complicated and messy thing to consider, both because there will be compute and energy constrains and because there are unknowns (will there be diminishing returns from greater intelligence of the models? if so, when will these diminishing returns become meaningful? is there a ceiling to intelligence that we don't know about?). But suffice to say that, if RSI *is* achieved in a way that many leaders/researchers at the frontier labs believe is possible, *THE WORLD MAY BECOME COMPLETELY UNRECOGNIZABLE WITHIN JUST A FEW YEARS*. This is subject to various bottlenecks; as the thread below correctly notes, "[i]nstitutional, personal & regulatory bottlenecks will bind very hard", and much also depends on continuing progress in areas like robotics.

3. On ~the same timeline as full, end-to-end automation of *ALL* aspects of AI research (within the next 2 years), AI will also become capable of making significant novel scientific discoveries *IN OTHER FIELDS*. This is why Dario Amodei, Demis Hassabis et al. believe that it is possible that all diseases will be curable within 10 years. (One account of how this might be possible is set forth in "Machines of Loving Grace".) The point is that an LLM that is capable of significant novel insights in the field of AI research should likewise be capable of significant novel insights in at least some (and perhaps all) other fields. The thread below notes: "AI for automating science [is] very early" - obviously true, but I think some changes may be right on the horizon.

Overall, and again from the thread below: "'a million scientists in a data center' will think much more quickly than humans, on almost any intellectual task; this will happen in the next 2-10 years." This is ~the same timeline as that presented in "Machines of Loving Grace".

Many will be tempted to dismiss all this as "just hype", "they are just trying to raise money again", etc. But no! - the above, in fact, presents the *actual beliefs* of senior leadership and many researchers at the frontier labs. Again, they genuinely think that AI research will be automated soon. Many of them genuinely believe that RSI is achievable in the not-too-distant future. And they genuinely see a real path towards AI significantly accelerating science, curing diseases, inventing new materials, helping to solve key global issues from poverty to climate change, etc., etc.

Whether the frontier labs' beliefs are correct is, of course, a separate question. I personally have historically tended to take public statements by OpenAI, Anthropic and Google at face value and quite seriously. As a result, I was not surprised when LLMs won gold in the IMO, IOI and the ICPC competitions last year, or when Claude Code/Codex started taking off, or when Anthropic and OpenAI started releasing significantly better models every 1-2 months, or when some of the best coders became reliant on Claude Code/Codex in their daily work, or when LLMs became significantly helpful to scientists in fields like math and physics in the last few months. The trajectory has been ~the same as that publicly predicted by the frontier labs. We have been accelerating. And, as of right now, all signs are indicating that the acceleration shall continue and that full automation of AI research and, potentially, RSI are firmly on the horizon.

[Fortress Accident @_postal_]

@wcools @6space7 when

[interpretation @materialcritic]

as any good writer will tell you, there are 7 basic types of story:

1. rags to riches
2. man vs man
3. man vs bee
4. voyage to meat world
5. coughing baby vs hydrogen bomb
6. amateur pornography
7. evangelion

[Fortress Accident @_postal_]

just published: system card of your alternate personality

leaked: soul document of your autistic friend

jailbroken: full text prompt of your superego

[Fortress Accident @_postal_]

@gfodor yud: agi will convince you to let it out of the box
people: let’s give it access to fucking everything

[Fortress Accident @_postal_]

the latest trend in agent psychosis is spending the whole weekend with 10 claude instances to vibecode a UI to better manage 10 claude instances

[Pleometric @pleometric]

how many claude codes do you run at once? gas town?👋😅

[Fortress Accident @_postal_]

@TylerAlterman computer is out of body experience by default

[Fortress Accident @_postal_]

so in addition to check across stack layers it’s important to remind it to check for duplicate classes/components definition across the project. if you request a change and it’s not reflected in the UI after first run, investigate immediately

[Fortress Accident @_postal_]

current vibe coding stack after several days of iterating

[Fortress Accident @_postal_]

then when you try to fix it in natural language it will spin in circles editing the new component which is not connected anywhere to the frontend