Sven Schmit

Data Science @ OpenAI. Prev. @ Eppo, Stitch Fix, Stanford.

[Sven Schmit @spschmit]

How come Bayesian and frequentist guarantees can be so different, even when the results are the same? In 1994, Philip Dawid published a fantastic paper about this.

Let's look at this from an experimentation perspective!

https://t.co/SJu1Lyxonh

[Ramon Dario Iglesias @RamonDarioIT]

We are building Allyon: Battlecry, the first monster trainer where creatures actually learn from you. It's closer to the Pokemon anime than the games: Allyon understand your voice, learn bespoke behaviors, have personalities and memories, and form lasting bonds with you.

1/7

[David Holtz @daveholtz]

In the January/February 2025 @HarvardBiz (and online now): a new article that explains the steps that companies should follow to get better at experimentation

https://t.co/koAnw1Rat1

from me, @IavorBojinov, @rameshjohari, @spschmit and Martin Tingley

[TechCrunch @TechCrunch]

Eppo lands new cash to grow its app, website and AI experimentation business https://t.co/ChhuS23ACR

[Sven Schmit @spschmit]

@jxnlco or you just fitted a linear trend

[Sven Schmit @spschmit]

@jxnlco In your bio?

[Sven Schmit @spschmit]

@jxnlco Sounds familiar

[jason @jxnlco]

Fuck it ship it.

I wrote this while kinda being annoyed by people asking for advice.

But know if that I sound angry and haughty, it’s only 40% intentional.

https://t.co/2T4vSAnMqv

[Ramon Dario Iglesias @RamonDarioIT]

The last couple of months I've been working on a game called Thus Spoke Zaranova. A game where you as a human must pose as an AI. (1/7)

[Sven Schmit @spschmit]

@seanjtaylor And 💯 on priors are great; or rather they are awesome. Bayesian experiment analysis with "proper" priors is very compelling imo. Not leveraging them is throwing away the baby with the bathwater, which might as well be the tl;dr of the blog post 😀

[Sven Schmit @spschmit]

@seanjtaylor Thanks for the kind words! Thinking of the MDE in a power calculation as some sort of prior is interesting -- I have always thought of them more as a "worst-case analysis" which feels very frequentist compared to the "average case analysis" of Bayesian analysis.

[Sven Schmit @spschmit]

@jxnlco Learn Phoenix LiveView :)

[Sven Schmit @spschmit]

@jxnlco When thinking about Json, I think about Jason

[Sven Schmit @spschmit]

@teej_m @paul_rietschka This! The polars API is much nicer. The pandas API makes me want to write SQL

[Sven Schmit @spschmit]

@teej_m I'd say this is more an issue of using ratio metrics rather than averages; it's often unclear if a ratio metric going up (on average) is good or bad. Histogram doesn't really help here (a shift to the right can still be both good and bad)

[David Holtz @daveholtz]

We are now accepting submissions for @CODEConference 2023!!!

We welcome any and all research related to digital experimentation, whether it comes from academia, industry, or somewhere else. If you have any questions about whether your work is a good fit, shoot me a DM/email.

[Sven Schmit @spschmit]

@Chris_Said That p-value does not inspire too much confidence in that point estimate though; some quick math suggests that the corresponding 95% confidence interval is -50 to 260

[eppo @eppohq]

When does it make sense to bandit, and when to experiment? Let @spschmit walk you through the decision-making process https://t.co/wlNvQZHLpG

[Sven Schmit @spschmit]

Fantastic talk, highly recommended!

[Sebastien Bubeck @SebastienBubeck]

Last couple of weeks I gave a few talks on the Sparks paper, here is the MIT recording!

The talk doesn't do justice to all the insights we have in the paper itself. Neither talk nor twitter threads are a substitute for actual reading of the 155 pages :-)
https://t.co/OF45HHpoaY

[Sven Schmit @spschmit]

@PhDemetri @colin_fraser @Get_Eppo On it!

[Sven Schmit @spschmit]

@hspter @Chris_Said ^ this