Evrenn

I break brains and chatbots with math • Same traps fool you and AI

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

This is the explanation of fear vs real risk I wish everyone saw. Cows killed at least 108 Americans in five years, sharks killed 4 in six, yet the shark scares everyone more, and the model behind ChatGPT makes the same mistake.

Your brain judges risk by how easily examples come to mind, and shark attacks are vivid, rare and all over the news. Real risk is simple: how often it happens times how many people are exposed. Nobel winner Daniel Kahneman and Amos Tversky named this trap 50 years ago.

11 minutes. bookmark & watch today. run one scary headline through the formula and watch it fall apart

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

No one explains doubling better than this. Fold a sheet of paper in half 42 times and it would reach past the Moon, and early versions of the AI behind ChatGPT refused to believe it too

Every fold doubles the stack, so 42 folds make a 0.1 mm sheet about 4.4 trillion times thicker, but our brains guess in straight lines and stop at "a book" or "a desk". That's also why Warren Buffett had $12 billion at 65 and still made more than 90% of his fortune after that

10 minutes. bookmark & watch today. make your guess before you watch. It will be wrong

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

This is the explanation of the wisdom of crowds I wish I'd had years ago. 787 strangers at a country fair guessed an ox's weight, most were off by 50 pounds or more, and their average missed by just 1 pound.

When people guess independently, their mistakes point in different directions and cancel out, and the more they disagree, the smarter the crowd gets. AI models learn from similar data and make the same mistakes, so asking five chatbots often gives you one opinion five times.

10 minutes. bookmark & watch today. a crowd is only as smart as its disagreements

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

There is no better way to understand why yelling at people seems to work. A flight instructor told a Nobel winner that screaming made his pilots fly better and praise made them worse. He was right about the pattern and completely wrong about why.

Every result is part skill and part luck, so an amazing day is usually followed by a normal one, and so is a terrible day.
Praise the great day and the next one looks worse, yell at the bad one and the next looks better - and in one test GPT-4 still blamed the wrong cause four times out of five.

11 minutes. Bookmark & watch today. The hidden reason your best streaks always seem to end right after someone notices.

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

There is a coin game whose average payout is infinite, and almost nobody will pay even $25 to play it. Every expected-value calculator, every betting model and every AI that prices risk by averaging runs straight into it. The math says pay anything. Every human says no.

Expected value adds probability times prize. Heads on the first flip pays $2 with chance 1/2 - worth $1. Heads first on flip two pays $4 with chance 1/4 - another $1. Every term is exactly one dollar, and the list never ends. In 1738 Daniel Bernoulli fixed it by averaging the logarithm of the prize instead, because a dollar matters less the more you already have. Counted that way, the whole infinite game is worth $4. Add the fact that no bankroll is infinite and even $79 billion behind the table caps the fair price near $37. When researchers gave the puzzle to 28 AI models, some still answered infinity.

10 minutes. Bookmark & watch today. The logarithm trick that turned infinity into four dollars.

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

Flip a fair coin three times and look at every flip right after a heads. The expected share of heads is not one half. It is 5/12, about 42%. The coin is fair. The averaging is not. Every streak statistic, every per-episode AI benchmark and every average of averages carries the same hidden bias.

The mechanism is averaging a ratio. For each sequence the share is heads-after-heads divided by flips-after-heads. Average those shares and every short sequence gets one vote no matter how many flips it holds, so heads quietly disappear. Pool the counts first and you get exactly 4 out of 8. This error sat inside a famous 1985 study for 33 years and made a real hot hand look like an illusion. LLMs asked to generate coin flips make the mirror mistake - their sequences come out too tidy, with long runs missing.

11 minutes. Bookmark & watch today. Write out the eight sequences and you’ll never trust a streak stat again

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

@Meddes0119 thanks, I’d appreciate it if you subscribed

[Evrenn @0xEvrenn]

When researchers at the University of Illinois ran coin-flip bets on Anthropic's Claude 3 Opus, it weighted a loss more than six times as heavily as an equal gain. A typical human comes in at about two.

That human number comes from Daniel Kahneman and Amos Tversky. In 1979 they showed that losing $100 hurts about twice as much as winning $100 feels good, and Kahneman won the Nobel Prize for the work. Their formula makes a perfectly fair coin flip feel bad, and most people won't take a 50/50 bet to lose $100 unless the win is around $250. It costs real money. In about 10,000 brokerage accounts, investors kept selling their winners and sitting on their losers, and the winners they sold went on to beat the losers they kept by about 3.5 points the following year.

In the Illinois study, ChatGPT came out milder than a typical person and Gemini landed almost exactly on Kahneman's number. A chatbot trained on human writing can hand your own bias back to you the moment you ask it whether to sell.

10 minutes. Bookmark & watch today. A Nobel economist's trick in the middle turns a bet you'd refuse into one you'd take 100 times.

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

Years before Daniel Ellsberg leaked the Pentagon Papers and faced up to 115 years in prison, he designed a $100 bet that trips up most people. A jar holds 90 balls: 30 red, and 60 black or yellow in an unknown mix.

Most people prefer betting on red over black, then prefer black-or-yellow over red-or-yellow. No single guess about the jar can make both choices consistent. Ellsberg published it in 1961, and it showed that people dislike unknown odds even more than bad odds. Researchers gave a question built on his bet to GPT-4 100 times, and it gave the human answer all 100 times.

10 minutes. Bookmark & watch today. Try both bets on yourself before you watch, then on any chatbot

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

Most doctors get this wrong. In 2007, psychologist Gerd Gigerenzer gave this exact mammogram problem to 160 gynecologists, people who read these results for a living. Nearly half said 9 in 10. Only 21% picked the right answer, 1 in 10. Back in 1978, 27 of 60 doctors and students at Harvard Medical School answered 95% to a similar question. The truth was about 2%. The trap is the base rate: when a disease is rare, the healthy people produce far more false alarms than the sick people produce true ones.

AI runs on the same math. A fraud detector that’s 99% accurate, hunting one fraud in a thousand transactions, flags about ten real frauds and about a hundred honest people. Most of its alarms are false. GPT-3 aced the famous cab version of this puzzle. Then researchers asked about a black cab that wasn’t in the problem at all, and it still answered 20%.

6 minutes. Bookmark & watch today. One hour of counting people instead of percentages took doctors from 21% right to 87%

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

@larwick the two errors don’t cost the same in a field, a missed weed eats yield all season while a sprayed crop plant costs you one plant, so the knob probably leans toward recall

[Evrenn @0xEvrenn]

Around 2015, an MIT graduate student named Joy Buolamwini built a mirror that was supposed to find her face. It couldn’t, until she put on a white mask. So she built a fair test: 1,270 politicians from six parliaments, split into four groups by skin type and gender.

Microsoft’s face software scored 93.7% overall. For darker-skinned women it was wrong about one time in five. IBM missed more than one in three darker-skinned women and only three in a thousand lighter-skinned men. The headline number was true. It was just an average, and each group’s weight in it was set by how many of its faces were in the test. Language models get ranked the same way today, with one score averaged over dozens of subjects.

11 minutes. Bookmark & watch today. One line of school math shows how a 96% system can fail a third of a group.

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

@0xorenx was the reactor already unstable from xenon buildup hours earlier, or did the test itself push k over the edge?

[Evrenn @0xEvrenn]

@0xzivo does anyone audit the discount rate these days, or only the cash flow forecast?

[Evrenn @0xEvrenn]

@0xBenjamine who chose the discount rate on the blockbuster deal? if enron’s own team picked it, what was andersen checking?

[Evrenn @0xEvrenn]

@0xzivo that $630k is 63 bps, which is what σ·√(Q/V) gives if the stock moves 2% a day and the order is 10% of daily volume

[Evrenn @0xEvrenn]

@0xBenjamine @georgesoros wild part is britain did better after losing. rates came down once the peg was gone and the recovery started, which is why some economists call it white wednesday instead

[Evrenn @0xEvrenn]

@leopardracer are the 10 calls per ball fixed for every model, or does each one decide how long to think? 20.9s per throw looks like gemini is burning way more tokens per call

[Evrenn @0xEvrenn]

in 1946 a statistician named Joseph Berkson discovered a statistical trap that later made cigarettes look like COVID protection even when the data could not prove it

keep only cases where A + B ≥ a cutoff and two unrelated things suddenly look like opposites

he spotted it in hospital records where one disease seemed to protect you from another

in 2020 smokers looked oddly rare among COVID patients and French researchers were reported to line up nicotine patches

a model where smoking does nothing still shows 18.1% COVID among tested smokers vs 21.7% for non smokers

same math is why the best restaurant in town sits on an ugly street next to a car wash

a team from Meta AI and Max Planck tested GPT 4 on this trap and it mostly guessed

11 minutes, save it and watch before you trust any chart built on filtered data ↓

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC

[Evrenn @0xEvrenn]

Sydney's $7 Million Opera House Became a $102 Million Disaster Because Everyone Ignored Daniel Kahneman's Forecasting Formula

the sydney opera house was budgeted at $7 million, it ended up costing about $102 million, more than 14 times the plan, and opened roughly a decade late

nobel winner daniel kahneman caught the same mistake in his own team, they figured a textbook would take two years, a colleague pointed out that similar teams took 7 to 10 years and many never finished, they kept going anyway, and it took eight years and was never used

students do it too, they guessed their thesis would take 33.9 days on average and it actually took 55.5

the fix is reference-class forecasting, forecast = base rate + w × (your estimate − base rate), where the base rate is how long similar projects really took and w is how much you trust your own gut, close to zero unless you have hard evidence your project is different

AI agents have the same problem, they'll hand you a clean timeline with ten milestones and exact dates that's really just a confident guess, so give them the real durations of past tasks like this one and only adjust when the new task is clearly different

the same mistake turns a startup's six-month roadmap into a year of burning cash, so check the base rate before you trust the plan, especially when the deadline controls your budget, runway or next round

bookmark this before accepting an AI agent's deadline or promising a client a delivery date ↓

[Evrenn @0xEvrenn]

ai won gold at the math olympiad and still invents fake court cases that get lawyers sanctioned

big tech is betting trillions on a machine rewarded for guessing so read this before you trust it ↓ https://t.co/8nm89PsbvC