Here is the reason WHY this happens. In Generative AI the text is split into tokens, each token is mapped to a vector (an embedding), and then the model reasons over those vectors. The spelling problem starts at step one. "Seventeen" might arrive as one or two tokens, and once that split happens, the letters are gone. The token is just an ID number.
The vector for that token must carry everything the model knows about it: meaning, grammar, context, and, only indirectly, how it's spelled. Spelling isn't stored as a literal sequence of letters in the vector. It's a fuzzy statistical association picked up from seeing the word near things like "s-e-v-e-n-t-e-e-n" or rhymes or misspellings in training data. So, the vector "knows" the word has a lot of e's but not with the precision of reading the characters.
So: tokenization removes the letters, and vectorization stores only an approximate memory of them. Letter counting is essentially a task AI must reconstruct rather than perceive. That's why spelling the word out one character at a time helps, because each letter then becomes its own token and AI can count what's in front of it.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
a few of you have been riled up by this video which includes such iconic phrases as AI “sucks balls at coding,” “go fuck yourself,” “materialist propaganda wet dream bullshit,” passing AI through a “glory hole,” and lastly, failing to properly take accountability for my children and mother-in-law.
for posterity, i’d like to make a written record of actual claims i’ve made without the seemingly not-so-funny shit-posting wrapper.
1. there is no consensus on intelligence in animals. why are armchair thinkers claiming AGI is already here? and why is demis hassabis meanwhile saying we are nowhere near AGI?
2. often when the shortfalls of LLMs is brought up, proponents say “yeah but, 12-18 months these issues won’t exist anymore.” this constant prognostication about future progress is not useful discourse and is a form of religious psychosis. i say that in future videos, i will simply judge llms by their current abilities, and not by future potential.
3. i say that llms are not actually intelligent and that david deutsch argues intelligence and consciousness/personhood seem to be inseparable. i say that for this reason llms are not actually intelligent but “trick” credulous people into believing they are. here of course we are getting into axiomatic word game territory so no one wins. that feigning intelligence is equivalent to it is not a settled matter and while this “fake” intelligence may help you do some work, it does not obviate the need for clearly delineating between true intelligence and a model of it. actually nothing is more important than this very question.
regarding ai “sucking balls at coding”, i was somewhat re-enlivened to make this claim after watching a 20 minute video i uploaded on yt back in february when i used to do coding screen shares / daily dev logs. i was (am) a huge user of these tools. the “sucks balls” isn’t that it can’t do some things for you. it’s that it can’t ultimately get you to your final final destination. to me, it wasn’t so obvious there were net gains in productivity, all things considered, even though day to day you kind of felt like there were.
i make about 2-3 videos a week and it’s easy to miss that these videos are usually performative storytelling. they are presentations meant to get you through the whole thing. that this one was particularly over the top was precisely the point. it’s a “drunken rant” genre that is meant to be a vicarious/cathartic ventilation of pent up frustrations that many of us feel towards the industry-wide token mania that is resulting in the serious loss of what it even means to be human. and for what? higher productivity gains? my job is simply to tell stories that draw attention to this fact.
@steipete It took me 3 days (off & on) to watch the @LexFridman interview. Superb. Visionary. Joyous! I have been in cybersecurity and tech > 30+ years and remember 2 other times that stopped me in my tracks and made my heart rush. The first was seeing the GUI on the net… I remember standing in a Comdex booth and meeting Marc Andreesson, when he was demoing Netscape Navigator and I watched as the page scrolled down slowly. I was blown away. The second was my first chat w/ ChatGPT in ‘22. I dropped my iphone on the floor and then called everyone knew screaming (the world is over as we know it)!!! Then I heard about OpenClaw. I ran out of my house, bought mac mini (typed one line in my terminal and waited. My heart was racing. Mostly because I knew I had to give it full access. And then there it was Ta Da!!! OpenClaw!! Ready to explore the world of Lobsters! I was (and am) hooked! No pun intended… lol Thank you sooooooo much for being so AUTHENTIC in the interview. But most of all thank you for creating OpenClaw. My life will never be the same. And I love that you gave it a heartbeat. Thank you Peter you wonderful HUMAN!!!!
@steipete@AlexFinn Thank you Peter for Calling this out. @AlexFinn is ROCK STAR and a huge supporter - but it is important t know what is really accurate. ALex - you should THANK Peter for taking the time to clear this misinformation up. It only makes you better!!! OPENCLAW is Insane. I love it.