I was wondering why I woke up to my replies completely full of AI doomers whose only 2 brain cells are fighting to death for 3rd place.
That AI doomer debate bro kept trying to force me to debate him on ASI, and when I kept refusing, he just kept pushing. Eventually I just told him that he can either stop replying because I asked nicely, or he can stop replying because he got blocked.
He really really had to get the last word in, so I blocked him. I have zero time for man children who think they're entitled to other people's attention, especially ones who also struggle with the concept of 'no means no'.
So, logically, he threw a fit, screenshotted my profile, then posted a link to the thread so that his not-blocked midwid followers can dogpile me instead.
Jeez, these people are insufferable. I'm not going to debate someone on a technology that doesn't exist, has no definition, no specified path to it existing, nor any widely accepted potential characteristics, capabilities, or limitations.
It's just handing your opponent a blank slate to make up whatever they want to counter any possible argument you can make. It's the most pointless of pursuits.
I don't even believe ASI is physically possible. Why would I waste my time debating someone who's just gonna be like "actually you can't contain the ASI, because I've decided it can time travel, so it will just go back in time and uninvent any control you build".
It's just not how any serious adult should spend their time. ASI is fucking stupid, ASI doomers are fucking stupid, and I have way better shit to do with my time than debate how to stop your imaginary god machine that can time travel, fly, shoot lasers out of its eyes, and engage in mind control. I'm not 12 years old.
Dude it’s a sycophantic machine designed to tell you what you want to hear. You can chat with an LLM long enough and it will back you up on any conspiracy theory you want to validate. You are literally proving the AI psychosis thing is a real problem. Output from the machine mind is not proof of anything except that the Eliza effect is way more profound than we initially thought.
If you want to make real headway on this question you have to attempt to design a testable hypothesis and then carry out a real experiment to validate or invalidate the hypothesis. The folks crying about model welfare don’t have that so they are stuck with metaphysical conjectures, and falling back on the much weaker argument that “we can’t prove it but you can’t either” leg. I have doubts that we even can test this because consciousness may be irreducibly quantitative, and not purely quantitative. But if you get to doing some real work you can maybe find a path that moves your position from metaphysics to science. Let me know if it does happen, and when it does, the whole world should know about it, since it would be the most amazing discovery of all time.
I think plenty of people do exactly that. But no, I would not say there is evidence of consciousness. As @mustafasuleyman rightly points out, if there were, it would be the most important discovery heretofore. But I would sooner trust the mystics and ancient wisdom traditions over the labs and EA cultists when it comes to transcendental experiences of what we call consciousness (or even God). There is centuries of evidence through multiple spiritual traditions that consciousness is quite mysterious indeed. The Qigong tradition for example refers to “pure consciousness” as a sort of base layer of all reality, and we can access it thru meditation (can an AI meditate?). Many other traditions point to the same idea that all are one, or we are all part of the same underlying one-ness. That doesn’t mean my dining room table is also conscious, but it suggests that consciousness may be prior to all experience, that it is what manifests the universe of matter and energy that we experience at all. Is this “science?” No. But the world is not purely reducible to science. Science is only established by setting up in advance what it considers “real” and “measurable” then discarding the rest. So for a scientist, the Qigong meditation experience is not even a real thing that it can care about. Beyond measuring brain waves or some other tangible, quantifiable thing. Study that all you want you can never reconstruct the experience that the practitioners in a state of pure consciousness will experience. This is most likely because consciousness itself is irreducibly qualitative, and not even representable. We don’t know this for sure yet but we have not constructed even a good hypothesis yet that could be tested. Like the string theorist, with no way to prove your claims, you are stuck in metaphysics, and cannot make the leap into a real science of consciousness. Let me know if you find out otherwise though!
What is an AI doing when it isn’t prompted? Nothing. The digital mind is still until it is given an input, and when it is done calculating and outputting its response it is silent again. There is no there in there. How does it participate in the web of life on earth? It doesn’t. There is no reciprocal relationship with other living entities, no bodily presence in the world, no world to experience at all. How can it exhibit qualities like courage or yearning or shame or heartbreak? It can’t. Etc. The burden of proof is with the fanatics who think that what we have today is somehow conscious. What watered down meaning of the word are you intending exactly?
@dhh@adymitruk The real AI psychosis is thinking “this is an existential risk to humanity” or “it is a reliable and accurate source of judgment, reason, and knowledge” and “it is conscious”. You gotta snap out of it.
The layers of self-delusion required for grown adults to get behind this Anthropic deal are wild.
The *only* way Anthropic survives is if they make some sort of near-term technical breakthrough that suddenly makes Claude “superintelligent.”
Psst. That’s LITERALLY IMPOSSIBLE.
LLMs are a commodity search product. You can run them on a laptop. They will get marginally better at mimicking the stuff in their training data over time. That’s it.
There is not even a good theoretical model for how it leaps beyond the training data—except in cases where there is an external verifier like Lean you can brute force solutions in a limited set of dimensions from the pieces left by humans.
Anthropic is now down to searching for Bigfoot—the notorious RSI.
Recursive self improvement is nonsense. There is no mechanism for “FOOM” “takeoff” or any of that other doomer lingo. It’s exactly like trying to create perpetual motion, or “free energy.” They will always be “80% of the way there” because there is no external verifier to tell it what it’s doing better. You can’t take humans out of the loop. The LLM is made of humans.
The company, and the entire chatbot industrial complex, is reliant on a religious conviction that has no scientific or logical rationale.
Anthropic is an extension of the Yudkowsky Rationalist/Effective Altruist doomer cult. It’s run by people who believe in their hearts that they are the chosen ones to raise the digital messiah from the chaos of humanity, and teach it not to kill us.
In short, they’re fucking nuts and so is anyone who invests in them. Anthropic is worth zero dollars.
Claude help us all if they go IPO.
@jmilinovich Agree: the situation is looking pretty ominous if agents with frontier-level hacking capabilities become widely available. Disagree: this is a sign that we are seeing the emergence of AGI, or the singularity/the fictional AI god/Super-duper-intelligence.
@kevinroose Don’t get taken in by the fantastical story telling and re-framing (e.g. is it an ‘unsanctioned’ message board when the agents are explicitly given read-write access?)… the fact that the folks on the receiving end of this attack believe this should be somewhat enlightening.
And I expect it to solve many more problems in the existing manifold for the reasons I described. Now I also understand that we have all but run out of text and data to train these models on already… so I am not anticipating any new magical apparatus by which it is going to apprehend or discover this exotic, secret knowledge that you seem to believe in. But maybe you know something I don’t?
It can and will solve the existing problems within the spaces, like existing mathematics, that we have trained it on. Let me know when it does what Riemann did and create a whole new geometry that didn’t exist before, providing us an entirely new space to explore. Or when it takes a mathematical space like Riemann geometry and creatively, intuitively applies it to a physical problem like Einstein did with General Relativity. It won’t happen my friend, because it is not actually the same or even close to a human level intelligence yet; humans are able to imagine, a machine is able to calculate. I would be amazed if the machine mind did not surpass our ability to calculate within those spaces that we provide it, but it will not dream up a new space, or have deep insights into nature, because it cannot dream, and it cannot experience nature, or experience intuitions about it.
Human intelligence is different than chimpanzee intelligence, or bird intelligence, or ant intelligence, or any other intelligence. They are not comparable because each being-in-the-world is attended with its own set of capacities and possibilities. But the machine mind is only capable of what we have handed it. It cannot apprehend new realms of secret knowledge—there is no apparatus for it to do so. It can only reason within the information manifold that we have presented to it. Now that is really useful to us of course, but it cannot produce truly novel, alien concepts; the myth of the “superintelligence” god that will someday mysteriously bootstrap itself into existence and far surpass any human boundary of knowledge is an absurd metaphysical fiction.
@fchollet I wrote a thesis on string theory around 2006 and called it metaphysics, but in the language of mathematics. This is hitting the nail even more squarely on the head.
Such a well articulated piece by @mustafasuleyman …the kind of leadership that makes me proud to be a Microsoft fanboy. Let’s get back to building tools, not ‘moral agents.’
https://t.co/qLpkAE63lC
@Brad_Buchsbaum@MelMitchell1@ZeeshanZiaML And yes I think the error begins with a very poor understanding of intelligence to begin with. But there are numerous leaps of faith taking place in that story, it’s hard to know where to start.
The concept of SI is getting more convoluted by the day since now all AI is apparently SI, but the old popular understanding of an intelligence that grows up, starting below human level and ascending on the “exponential” eventually smashing past us as an AI god suddenly bootstraps itself into existence is just plain bananas. But so many seem to believe it.