I tried to find the best stock callers on X using AI.
I read 90,856 posts (141M tokens), graded 8,000 calls against real prices, and tested 659 accounts for skill vs luck.
I couldn't find a single one that clearly beat luck.
How it works:
An AI model (Jev by @typesafeai) read every post and decided: real prediction or just news? Bullish or bearish? Timeframe? Price target? Then code graded each call on what the stock actually did next, vs its sector and the S&P 500.
Cost: 141M tokens for $5.19, 90,856 posts for $7.59. Built in a day.
Two traps almost fooled me:
1. Hindsight. My first chart said following the "best" accounts was right 80% of the time. But "best" was judged on the same calls I was testing. Using only what was known on the day: 45%.
2. Luck. The best record was 61% over 96 calls. But with 659 accounts tested, someone that "good" shows up by pure chance 94% of the time.
Result: the 62 most active accounts, pooled, were right 50.3% of the time. Chance: 49.8%. To prove someone is truly right 65% of the time you'd need ~235 independent calls, and almost nobody gets close.
What this does NOT show: direction isn't profit (good traders win with risk management), the AI can misread sarcasm or chart-only posts, and it's only 14 large caps in 2024–26.
I never expected to beat the best investors. I wanted to know if the loudest stock accounts on X actually know something. Not measurably.
Not investment advice. Tell me what I missed 👇
@TheProfInvestor This is basically my research in meme form 😄 I had an AI grade 8,000 stock calls from 659 X accounts. The best record looked like skill, but among that many people, a streak like it shows up by pure luck 94% of the time.
Without a good model it might still brute-force and reason to the correct answer.
But to get true innovation the model needs raw intelligence.
It’s like doing math where one person plugs numbers into a formula, while the other has the intuition for why that formula works.
Astra is strong on raw capability. For initiative, creativity, and design taste, Fable still wins sometimes. GPT is the clear winner on bang for the buck and actually getting things done though.