fabio casati

father of two

[fabio casati @sphoebs]

Enterprises are doing AI eval wrong - and it's causing wasted iteration cycles and wrong decision making.

I’m preparing my spring semester course on designing large-scale AI systems and I need feedback:

What’s the one change you’d make to eval practice + reporting to make it reliable?

What's the error you see made often?

[fabio casati @sphoebs]

@xAviation Where is he going.

[fabio casati @sphoebs]

@SawyerMerritt Omg that’s so 2020….

[fabio casati @sphoebs]

@nic_amadio This is very true. Italian engineers are massively underpaid. Companies that hire and pay well do wonders in Italy - and everywhere.

[fabio casati @sphoebs]

@ylecun @haider1 Why S?

[fabio casati @sphoebs]

@karpathy We als need new programming abstractions. A powerful one is the notion of statistical assertions. Properties that you’ve weakly expect to be true over your flow but not all the times.

[fabio casati @sphoebs]

@DanielMiessler “The best influencers”? What’s that category. The most competent or the most controversial or, like, Mr. Beast?

[fabio casati @sphoebs]

I keep seeing posts on AI and SaaS. My (biased) take is that AI agents need SaaS way more than humans do. And this is from experience.
Agents can consume knowledge like no human can. They can learn and iterate faster than humans. They can also go off track faster than humans—and at scale.
They need a platform that maximizes their potential while keeping controls.

[fabio casati @sphoebs]

@Lol19559014 @JenniferSey That is not what he said. That is a redacted selection. It’s ok if you are trying to help him get more supporters, because this is what’s posts like this do.

[fabio casati @sphoebs]

@petergyang Air way better than pro. Minimum 15. 14 is frustrating.

[fabio casati @sphoebs]

@_BMSimpson @curiosityonx It’s always 5pm where I am.

[fabio casati @sphoebs]

@Angaisb_ I don’t get all the people with “my agent takes 10 minutes”. Try the same simple ask to both opus and gpt. Difference is huge.

[fabio casati @sphoebs]

@IntCyberDigest Please do!! This idiotic healthcare privacy forced on people who don’t want it is preventing progress in medicine.

[fabio casati @sphoebs]

@lugaricano @mustafasuleyman @Microsoft You are asking for the moon. Let’s set a lower goal, like automating many routine elderly care tasks with robots.

[fabio casati @sphoebs]

@travisakers Why. Do you need to hide your cholesterol from the CIA?

[fabio casati @sphoebs]

@operationdanish Why not?

[fabio casati @sphoebs]

Anybody remembers the 2025 NY humanoid robots show ? Aside from dancing, I hope companies work on robotic care for older adults. I’d trust it more than nursing homes

[ChineseEmbassyManila @Chinaembmanila]

At the 2025 Spring Festival Gala, 16 humanoid robots joined a traditional folk dance known for its sweeping steps and vibrant handkerchiefs.

During the grand opening performance, humanoid robots demonstrated the "Thomas 360" stunt move. In a subsequent stage comedy, a group of AI-powered humanoid robots, along with a bionic robot, took on comedic roles. And in a separate martial arts performance, robots executed a series of high-difficulty movements.

[fabio casati @sphoebs]

@kmcannon ^^ this. This never happened before. At scale. Think about it.

[fabio casati @sphoebs]

@JoshDaws Very true. Nothing compares to it. Not even iPhone. Maybe the internet

[fabio casati @sphoebs]

Imagine how fun for people preparing presentations on red teaming AI ))