Dan Evets

AI/Generative Tech for building big things in the physical world. Cycling Geek; Jerry Colonna Fan; Former Pro Derivatives Trader

[Dan Evets @TuringTest_Dan]

@sportslarryk - I’ve really enjoyed listening to you on @KNBR again this week. Always great insights and thoughts. I don’t always agree with you, but I really appreciate your takes. You’re how sports radio should be. Keep up the great work, man.

[Dan Evets @TuringTest_Dan]

@nikitabier @AngelMD1103 ❤️

[Dan Evets @TuringTest_Dan]

@FrankCurzio Always great insights, Frank. Been listening to you for over 20 years and your track record is A+

[Abdul Șhakoor @abxxai]

BREAKING: 🚨 Someone just tested 35 AI models across 172 billion tokens of real document questions.

The hallucination numbers should end the "just give it the documents" argument forever.

Here is what the data actually showed.

The best model in the entire study, under perfect conditions, fabricated answers 1.19% of the time. That sounds small until you realize that is the ceiling. The absolute best case. Under optimal settings that almost no real deployment uses.

Typical top models sit at 5 to 7% fabrication on document Q&A. Not on questions from memory. Not on abstract reasoning. On questions where the answer is sitting right there in the document in front of it.

The median across all 35 models tested was around 25%.

One in four answers fabricated, even with the source material provided.

Then they tested what happens when you extend the context window. Every company selling 128K and 200K context as the hallucination solution needs to read this part carefully.

At 200K context length, every single model in the study exceeded 10% hallucination. The rate nearly tripled compared to optimal shorter contexts.

The longer the window people want, the worse the fabrication gets. The exact feature being sold as the fix is making the problem significantly worse.

There is one more finding that does not get talked about enough.

Grounding skill and anti-fabrication skill are completely separate capabilities in these models.

A model that is excellent at finding relevant information in a document is not necessarily good at avoiding making things up. They are measuring two different things that do not reliably correlate. You cannot assume a model that retrieves well also fabricates less.

172 billion tokens. 35 models. The conclusion is the same across all of them.

Handing an LLM the actual document does not solve hallucination. It just changes the shape of it.

[andrew chen @andrewchen]

prediction re the end of spreadsheets

AI code gen means that anything that is currently modeled as a spreadsheet is better modeled in code. You get all the advantages of software - libraries, open source, AI, all the complexity and expressiveness.

think about what spreadsheets actually are: they're business logic that's trapped in a grid. Pricing models, financial forecasts, inventory trackers, marketing attribution - these are all fundamentally *programs* that we've been writing in the worst possible IDE. No version control, no testing, no modularity. Just a fragile web of cell references that breaks when someone inserts a row.

The only reason spreadsheets won is that the barrier to writing real software was too high. A finance analyst could learn =VLOOKUP in an afternoon but couldn't learn Python in a month. AI code gen flips that equation completely. Now the same analyst describes what they want in plain English, and gets a real application - with a database, a UI, error handling, the works. The marginal effort to go from "spreadsheet" to "software" just collapsed to near zero.

this is a massive unlock. There are ~1 billion spreadsheet users worldwide. Most of them are building janky software without realizing it. When even 10% of those use cases migrate to actual code, you get an explosion of new micro-applications that look nothing like traditional software. Internal tools that used to live in a shared Google Sheet now become real products. The "shadow IT" spreadsheet that runs half the company's operations finally gets proper infrastructure.

The interesting second-order effect: the spreadsheet was the great equalizer that let non-technical people build things. AI code gen is the *next* great equalizer, but the ceiling is 100x higher. We're about to see what happens when a billion knowledge workers can build real software.

[Dan Evets @TuringTest_Dan]

@rabois @rabois - Happy New Year to you and the family.

Also: much respect for the solid workouts last year — you simply crushed it.

[🇦🇪 HGS @Sajwani]

Emirates 🇦🇪 made Sleigh380— sending Santa long-haul this year!

How ?? 😳

Merry Christmas from Emirates. 🎅🎄🎁

[Michael Dell 🇺🇸 @MichaelDell]

Vote FOR @elonmusk. The award is only achieved IF he hits exceptionally ambitious market-cap and operational milestones—if he falls short, he gets nothing. If he succeeds, shareholders will win big through unprecedented value creation, and he will earn added voting rights to continue driving @Tesla's long-term vision.

[Dan Evets @TuringTest_Dan]

What a great post by @bhalligan — brings things back to the basics. Sound advice in an era where it now seems like somehow , somewhere, CEOs of private companies were convinced into thinking that they should dazzle, or amaze — or channel their internal Steve Jobs — when meeting with their boards.

[Brian Halligan @bhalligan]

Board meeting tips for CEO's of private companies:
1. Be transparent -- it builds trust. Board members can tell if you are spinning them. Save the sales pitch for the sales prospect.
2. Prep time -- Either send slides or a memo at least 2 days before the meeting if you want your board to be useful.
3. Opening remarks -- Before you dive into the slides, have opening CEO remarks. I used to give my "mood" score on the business overall.
4. Roberts Rules of Order -- Don't worry about all the "I have a motion bs" in a private board.
5. Discussion -- The content should be designed to fill only half the meeting. A board meeting is a discussion more than a presentation.
6. Asks -- I used to bring "asks" for each board member (i.e. customer intro's).
7. 360's -- Once a year, do a 360 review of each board member. This will be good for them, but also give you a little power in the relationship.
8. Independent -- Get 1 truly independent, sharp board member to help and to balance the power dynamic.
9. Team -- Your senior team should play a part in the board meeting so the board can get to know them. Not all of them every time, though.
10. Culture -- Every board has its own culture. Don't be afraid to explicitly design and talk about it (i.e. lets not take ourselves too seriously).

I remember being nervous as sh-t before my first HubSpot board meeting. I hope this helps if you are feeling that way.

[Dan Evets @TuringTest_Dan]

@bgurley although I don’t always agree with your positions on certain topics, i’ve always appreciated your honesty, conviction, and sticking to your principles with rigor and intellectual curiosity. The pod won’t be the same without you, but I’m sure it will still be awesome. Hats off to you and much luck to you — an industry legend — in your next adventures.

[Jason ✨👾SaaStr.Ai✨ Lemkin @jasonlk]

So I recently caught up with a sales rep that made about $600,000 a year from 2020 to 2022. And then decided it was time for something new.

He's made about $150k a year since then, maybe a bit less.

What happened in his new roles?

Really just one thing: he'd forgotten how much easier it was to sell with a top tier brand behind him.

He's a good sales exec with start-up experience, but he did benefit from running his playbook for years with a top brand behind him.

When he left to a good opportunity ... but without a brand ... his W-2 comp fell from $600k to $150k.

There's more to the story (there always is) but in this case maybe not too much more. Same motion, same ACV, same industry.

But he'd just forgotten how different sales is with a strong, dominant brand behind you is.

And my advice to founders is: almost never hire a sales exec who has only sold with a strong brand behind them ... if you don't have a strong brand yourself yet.

[Dan Evets @TuringTest_Dan]

Great day and great week. Business is booming ! We’ve developed a generative AI platform w/aspects of combinatorial optimization to solve complex industrial construction projects —- specifically facilities within the three key industries that were highlighted in the @theallinpod AI summit: mining, energy, AI factories. It’s great to help re-industrialize America. Would love to connect with anybody else who is doing the same. 🇺🇸🚀

@Jason — the vibe shift is real.

[Dan Evets @TuringTest_Dan]

@lexfridman — hats off to you for the great, enlightening, fascinating, and thoroughly enjoyable interview with one of the OGs of AI, @demishassabis . As always, you have an uncanny ability to draw out the humanity and brilliance in your guests. 💡

[Dan Evets @TuringTest_Dan]

@PeterSchiff Says the guy who has been spectacularly wrong and bearish on the market for 20 years…takes guts .

[Dan Evets @TuringTest_Dan]

@united Thanks for the reply :)

[Dan Evets @TuringTest_Dan]

@united I really like you guys. Loyal flier, but increasingly every flight the planes are old, seats worn, bathrooms dirty. Are upgraded planes coming soon?

[Dan Evets @TuringTest_Dan]

@sundeep Why aren’t you using @grok ? Just sayin’….

[Dan Evets @TuringTest_Dan]

@tbpn Yikes!

[@jason @Jason]

I ordered 12 Optimus robots to give to my fans over the next 12 days… retweet to be entered.