Stephen Wolfram just posed the most disturbing thought experiment about AI, and nobody has an answer for it.
Wolfram: “Imagine humans are all in boxes. We’re all Darth Vader, inside these boxes, but you can’t actually see the human inside.”
Civilization continues identically. Every human hidden in a machine. You see the output, not the person.
Wolfram: “The world is operating, great paintings are being produced, but you can’t see any of the humans. All you see is a bunch of boxes doing human-like things.”
Civilizational Turing test. If the external world operates the same, are humans contributing anything essential?
Wolfram: “The world is operating as before, maybe even better than before. If you knew there were humans inside those boxes, you would say great outcome.”
That’s the paradox. Know humans are inside and it’s a golden age. Remove that knowledge and it’s just machines producing results. Does the value change?
Wolfram forces the question. Do we value creation or creator? If the art is identical, does consciousness behind it matter?
Wolfram: “You can’t tell there are any humans. It’s just a bunch of Daleks operating.”
From outside, machines behaving like humans look identical to actual humans. The show continues. Universe doesn’t register the difference.
As AI capabilities expand, this stops being abstract. AI produces indistinguishable art, music, science. Does human creation retain special status? Why exactly?
Wolfram isn’t answering. He’s exposing the void where our answer should be.
If outcomes are identical, is human involvement meaningful or just attachment to how things historically worked?
We assume human participation makes civilization valuable. If results don’t change either way, that assumption needs justification we’ve never properly given.
Real test isn’t whether AI replicates output. It’s whether we can explain why human output matters more when the results are indistinguishable.
If we can’t, we’re heading toward a future where civilization functions perfectly and whether humans are actually inside the boxes becomes irrelevant to everything except the humans wondering if they matter.
And at that point, are we necessary or just witnesses to a system that would operate identically without us, asking questions that have no impact on anything except our own sense of purpose?
To make it pedagogical, LLM is an autocomplete function that randomizes based on the statistical frequency:
+ Probability matching: if in the frequency space, the autocomplete is at 85% "take an umbrella" and 1/10^6 "have squid ink", it will do 85% of the time "take an umbrella".
+ "Temperature" setting: you can pick up to 100% the highest probability: "take an umbrella".
It's not China imitating the U.S. anymore. It is the U.S. imitating, and repeating the errors of Mao's "Cultural Revolution". The only good news is that it only took half a century for China to recover.
20 months later, evidence appears that Israel's claims were motivated disinformation, black propaganda to cover up how its own incompetent armed forces killed Israelis. The truth will eventually shine through. But not after the lie has covered up so many war crimes.