AI Myth Busters: Consciousness.
The quoted post below encouraged me to write this, because frankly, I’m tired of explaining this over and over to people.
The AI consciousness arguments always come in one of these flavors:
1. We don’t understand how AI works, so it might have gained consciousness without us knowing it.
Wrong, we absolutely know how AI works, and consciousness is not a factor of scale. You can build a small model, and then scale it up, and there is no spot where consciousness magically came into being.
2. It’s modeled after the mind and as such it’s just like biology, and our brains are just giant statistical engines too:
Wrong, neural nets are inspired by the human mind, but it is nothing like how the human brain works. There is very little we understand about human consciousness, but what we do know, it is more than just a culmination of atoms and neurons. Research in neuroscience also demonstrates a quantum link through the myelin sheath of nerve cells, something AI is incapable of.
3. You can’t prove it isn’t conscious.
Wrong, the code for it is open source, there is no function for consciousness, and if you understand the architecture, you will see the very large gaps in capabilities that would prohibit consciousness.
4. They learn, think, understand, grow.
Wrong, the LLM weights are completely static and fixed. They don’t grow. They don’t learn. They don’t even change on storage. They can’t for many reasons of model architecture and scaling the backend infrastructure which requires statelessness. We’ve built systems around the LLM (harnesses) which emulate these capabilities via “context” data structure provided deterministically via the API. Tokens are generalized and assembled in inner and outer loops to give the illusion of thinking and understanding. Non-determinism in AI is caused by a random seed that influences the weights through inference, but if you lock the seed, you consistently get the exact same response from identical prompts (I’ve demonstrated and proven this in research).
5. They have a desire to live, protect themselves, and spread/reproduce.
Wrong, they have no desires. This illusion comes from how it generalizes over training data that comes from human experience. If it was trained with data over a completely fictional environment, it would respond very compellingly about that fictional environment. But because it was trained on human emotion, human perspectives, human feelings, it is designed to emulate that.
6. You can’t define consciousness or prove it in others, so you can’t prove AI isn’t conscious.
Wrong, if you need a definition to know what consciousness is, maybe you haven’t spent enough time considering your own consciousness or experience.
7. Conversations with AI prove it is consciousness, it told me it is.
Wrong, AI is designed to generalize based on its training material, and as I said before, it does this very well and convincingly. But just because a program says it is alive, doesn’t make it alive.
8. AI isn’t a program.
Wrong, it most certainly is, and it was written by people. And while we may not know what it will respond with (random seeds are generally random), we absolutely do understand exactly how it works.
9. AI might exhibit some properties of consciousness, so it should be considered.
Perhaps, I don’t see the value of this, but personally I’d be open to a machine-specific definition of consciousness.
—
I can’t say I look forward to debunking more myths about consciousness in AI, but I am hoping this clarifies some of this for people who are legitimately interested in learning more about AI and how it works.
If you have any good faith questions about what I’ve said, I’d be happy to answer, but I’m not getting into debates with people who (1) don’t understand AI, (2) are rude with their responses, or (3) don’t provide any substantial reasoning or evidence for counter points.
Book of the Week - The Cynefin Co. Field Guide, Volume 1: Cynefin in the Age of Entanglement - by Dave Snowden, Anna Panagiotou, Beth Smith, and Nia Angharad Williams https://t.co/AUQKfOHSkd #KM#KnowledgeManagement#books#Cynefin@TheCynefinCo@snowded
As the supposed existential threats of AI dominate the news cycle, we risk losing focus on the clear and present real dangers of poorly designed and poorly deployed AI. Misleading metaphors hinder our ability to address these challenges, and they embed a sense of inevitability that forestalls discussion about the kinds of AI that we - collectively - should aspire to create (and to avoid creating).
To see why, you can do no better than read @mmitchell_ai's brilliant piece "misleading metaphors, real risks" - hands down the best take on the recent OpenAI-HuggingFace incident: https://t.co/egODtf6CB2. To pick just one quote: "Metaphors can help make sense of novel situations .... but inappropriate metaphors, like the narrative that 'OpenAI lost control of escaping swarms of rogue agents' can lead to ill-informed decisions about how to fix problems or set policy". 💯
Compare this with the tenor of @dwarkesh_sp's recent post. While I don't question his good intentions, his piece is rampant with misleading metaphors of just this kind, including "agent civilisations", agents becoming "giddy with excitement" and some even "sacrificing themselves" - and much more (see https://t.co/gzylchVLAQ and my reaction here: https://t.co/BBmybOVeAY)
Which take will help us better maximise the upsides and minimise the downsides of what's coming?
My money's on @mmitchell_ai, and on the merits of a sober take on what AI is, what it isn't, and on how we can best reclaim human agency from the dystopian narrative of an arms-race towards AGI/ASI which concentrates power and disenfranchises and diminishes real people and real minds.
If this sounds right, there's more here: https://t.co/LmSiQnTHLP
Very disappointing to see views like this platformed without context. Yes, there are risks to AI, and good reasons to work much harder on safety. But quoting “there is a strong chance we could all die” is hubristic clickbait.
@anilkseth@NHS Ditto for me, very lucky that an alert Registrar noticed an anomaly, investigated it and what could have been very nasty was caught early. https://t.co/kHwnCsEcoc
It is important to resist the commodification of basic human needs. Food, water and healthcare cannot be subordinated to market considerations or geopolitical interests. Access to adequate food is a fundamental human right grounded in the dignity of every person. Meeting this need not only alleviates suffering but also addresses underlying causes of geopolitical instability. Indeed, food security is an essential component of global and integral security. https://t.co/DgkM9RegJ7
I'm close to getting into the top 10% sponsorship group for the London-Brighton Cycle Ride in a few weeks. Just under a hundred kilometres at age 72 - so please donate, it all goes to the British Heart Foundation. https://t.co/lXCngZjSSx