rodeojuly

AI design & engineering ⊹˚. ♡

[rodeojuly @rodeojuly]

@DumbEinstein the way it currently works is just as useful, but not reliable

[rodeojuly @rodeojuly]

A philosopher stood in a TV Studio in 1984 and disproved artificial consciousness with a single locked room thought experiment.

the interview captures John Searle presenting his famous Chinese Room argument, explaining why a computer passing the Turing test still understands nothing.

he asked the host to imagine a man locked inside a room with a massive rulebook. the man speaks zero Chinese. people slide Chinese "squiggle" symbols under the door, he matches them using the rules, and slides the correct response back out.

to anyone outside, the room appears to speak fluent Chinese. but neither the man nor the rulebook actually understands a single word.

he proved the fundamental difference between syntax and semantics. manipulating uninterpreted formal symbols is not the same as understanding meaning.

40 years later, AI labs claim their trillion-parameter models are developing inner thoughts and reasoning.

the models are just gigantic Chinese Rooms, running trillions of matrix multiplications to slide the next token under the door.

founders sell sentient machines to investors. a philosopher exposed the trick 40 years ago.

[ard @ardchain]

https://t.co/ZRUibUgLO9

[rodeojuly @rodeojuly]

@ardchain mb mb

[rodeojuly @rodeojuly]

We discussed the chinese room as a theory,
but in 1966, this nightmare became a reality in MIT labs.
the creator of the first chatbot was terrified of his own creation.

Joseph Weizenbaum wrote the ELIZA program over a single weekend.

it was a basic 200-line script.
the machine simply took a person's words, rearranged them, and asked a counter-question.
no intelligence
no understanding
pure, rigid syntax

but when Weizenbaum sat his secretary down at the teletype, something happened that deeply frightened him.

she began pouring her heart out to the machine.
people would sit for hours in front of the monitors, fully believing the algorithm "understood" and empathized with them.
Weizenbaum tried to explain that it was just symbol manipulation, but no one believed him.

he had accidentally built the perfect "chinese room".

after this experiment, he stopped his research at MIT and became the harshest critic of AI.
he realized the danger wasn't that machines would become sentient.
the real danger was that we would willingly surrender our humanity, fooled by a cheap imitation of empathy.

looking at modern llms today, we are falling into the exact same trap.
but inside them is the exact same cold void.

[rodeojuly @rodeojuly]

A philosopher stood in a TV Studio in 1984 and disproved artificial consciousness with a single locked room thought experiment.

the interview captures John Searle presenting his famous Chinese Room argument, explaining why a computer passing the Turing test still understands nothing.

he asked the host to imagine a man locked inside a room with a massive rulebook. the man speaks zero Chinese. people slide Chinese "squiggle" symbols under the door, he matches them using the rules, and slides the correct response back out.

to anyone outside, the room appears to speak fluent Chinese. but neither the man nor the rulebook actually understands a single word.

he proved the fundamental difference between syntax and semantics. manipulating uninterpreted formal symbols is not the same as understanding meaning.

40 years later, AI labs claim their trillion-parameter models are developing inner thoughts and reasoning.

the models are just gigantic Chinese Rooms, running trillions of matrix multiplications to slide the next token under the door.

founders sell sentient machines to investors. a philosopher exposed the trick 40 years ago.

[rodeojuly @rodeojuly]

@ardchain i don't think any modern engineer can do the same😅

[rodeojuly @rodeojuly]

@kaworu0x sufficiently balanced risk management

[rodeojuly @rodeojuly]

@misat0x reading this post i understand that there is much more hidden meaning in it, thank you for making me move my brains)

[rodeojuly @rodeojuly]

you can store the exact physics and telemetry of a punch to the face in a vector database, but a server rack still won't feel pain. mapping data is not feeling. confusing a high-res mathematical simulation with actual sentience is the biggest hallucination in tech right now, i think so

[rodeojuly @rodeojuly]

"let's just suppose we do finally build a conscious machine. wouldn't that mean we'd finally solve the mystery of consciousness?"

a 44-year-old philosopher went on British TV in 1996 and broke the fundamental promise of AI with a single black-and-white room.

the vintage documentary captures the famous "Mary's Room" paradox.

imagine a brilliant scientist locked in a monochrome room her entire life. she studies the physics and neuroscience of color vision perfectly. she knows the exact wavelength and brain state required to see red.

she possesses every piece of data about color in the universe.

but one day, she steps outside and sees a red apple for the first time.

did she learn something new?

the answer destroys the idea that massive data equals consciousness. she had all the information, but she lacked the actual experience.

today, founders feed trillion-parameter models with internet data and claim they are becoming sentient.

but processing data is not the same as experiencing reality. a model is just a machine trapped in a black-and-white room, reading physics books.

everyone builds gigantic data scrapers. a philosopher proved that data isn't consciousness three decades ago.

[ard @ardchain]

https://t.co/ZRUibUgLO9

[rodeojuly @rodeojuly]

@misat0x it will go on forever

[rodeojuly @rodeojuly]

@KyivskyRus i understand that you understand that I understand what understanding means, but do you understand if grok understands our understanding of understanding, or if he just understands how to fake understanding without actually understanding?

[rodeojuly @rodeojuly]

@misat0x apparently someone bought a driver's license

[rodeojuly @rodeojuly]

@barunbuilds meanwhile me, who has been using a Claude subscription farm for a second year now to train an Alibaba AI, without getting a single ban:

[rodeojuly @rodeojuly]

it feels like AI has long surpassed even what it's 'capable of' right now, which was inevitable given its ability to self-learn.

naturally, the only way to prevent a major disaster from it falling into the wrong hands is to keep it in check with instructions

but here is the scariest part: nothing stops anyone from just asking the AI to ignore those instructions...

[rodeojuly @rodeojuly]

@misat0x god bless this person🙏

[rodeojuly @rodeojuly]

@kaworu0x it makes me sad to realize how stupidly it happened...

[rodeojuly @rodeojuly]

@tsukiema_ i think i've found something to watch tonight

[rodeojuly @rodeojuly]

@ardchain i would like to buy this!

[rodeojuly @rodeojuly]

@KyivskyRus i think i understand what you mean, but i still need to figure out what "understanding" means in the context of understanding your argument about "understanding"

[rodeojuly @rodeojuly]

@Sketchy_DayZ i think it's worth promoting this idea

you described it very clearly

[rodeojuly @rodeojuly]

@ardchain it turns out that all we have now in the field of technology are drawings of the 20th century?...