404

building at the edge of AI & quant

[404 @0xQuant404]

https://t.co/JXO4WCA0mP

[404 @0xQuant404]

OpenAI's ChatGPT, the most famous chatbot on the planet, couldn't count the r's in strawberry, it confidently said two, the screenshots went viral, and then Reuters revealed OpenAI's secret reasoning project was literally codenamed Strawberry before it shipped as o1

OpenAI built a whole $200 a month ChatGPT Pro tier around that reasoning, yet the model still never sees letters, GPT-4's tokenizer turns the word in your question into one single token, number 73,700, and that number is all it gets

OpenAI co-founder Andrej Karpathy says tokenization is at the heart of a lot of weirdness in LLMs, GPT-4 reads 12,345,678 as 123, 456, 78, a Reddit username once got its own token and made ChatGPT talk about "distribute" instead, and the same sentence in Ukrainian costs double the tokens of English

5 minutes of math to watch before you pay for another million tokens, so bookmark it and watch today because the 3 tricks at the end make the strawberry problem disappear in any model, ChatGPT, Claude or Gemini

[Rahul @sairahul1]

https://t.co/6qmUhlYMBw

[404 @0xQuant404]

@sama my rule is that if i can't explain why an answer is right, i haven't checked it yet. using these models every day made me more skeptical of them

[404 @0xQuant404]

@rohanpaul_ai broadcom grows by buying more than anyone on this chart. vmware alone cost $69b, ten times what nvidia paid for mellanox

[404 @0xQuant404]

@karpathy trying asd-ste100 for my video narration scripts now. short plain sentences sound a lot better read out loud