Top Tweets for #experimentguide
Slides for my KDD talk today on False Positives in A/B Tests: https://t.co/nDHRUioZ24
The paper is at https://t.co/v1q733K0FA
#abtest #experimentguide #successRate
The Imperfections of A/B Testing: https://t.co/fjNnLbURTS [doc]
The document addresses the 18 issues that were claimed to be serious issues with A/B testing. Some are valid, most are not.
Love to get feedback - doc is open for commenting.
#abtesting #experimentguide #abTestingIssues
Image credit to ChatGPT: draw an illustration of a person with a lab coat running an ab test with two tubes but the tubes have a flaw leaking liquid
![ronnyk's tweet photo. The Imperfections of A/B Testing: https://t.co/fjNnLbURTS [doc]
The document addresses the 18 issues that were claimed to be serious issues with A/B testing. Some are valid, most are not.
Love to get feedback - doc is open for commenting.
#abtesting #experimentguide #abTestingIssues
Image credit to ChatGPT: draw an illustration of a person with a lab coat running an ab test with two tubes but the tubes have a flaw leaking liquid](https://pbs.twimg.com/media/GLFX7zybIAAXTq2.jpg)
P-values have been misinterpreted for years.
Here is a summary doc with five of the best modern references I could find, with excerpts and annotations: https://t.co/Im2yC6rjPH
#experimentguide #pvalue
What's the cost of false positive #abtests?
It's not the immediate impact, but the long-term effect on the roadmap, guiding the ship in the wrong direction.
See more at https://t.co/PUD2g9v5Tt
#abtest #experimentguide
Why 5% should be the upper bound of your MDE (Minimum Detectable Effect) in A/B tests: short note I wrote at https://t.co/r904kfQUoT
#abtest #experimentguide #MDE
My talk at Conversion Jam 2023 on Causality and Pitfalls:
https://t.co/ibx7JCDLAe
If you want to learn more, my next Maven class starts Nov 6: https://t.co/MCJCj4EcFg
#abtests #experimentguide #CJAM2023
Be aware that at least some of the results (e.g., priming) in Thinking Fast and Slow by Kahneman are now thought of as highly exaggerated due to low statistical power. Here's a slide from my ConvEx '23 talk https://t.co/npxX08CWyz
#abtesting #experimentguide
“Judge me by how good my good ideas are, not how bad my bad ideas are.”
-- Matt Damon quoting Ben Affleck in his MIT commencement address at MIT, not realizing he’s talking about A/B Testing.
https://t.co/PkBCUWNSlV
#ABTesting #experimentguide
My list of the best five books for data-driven enthusiasts, and believers in Twyman’s Law is at https://t.co/lOPPfnwoT0
#datadriven #experimentguide
@Shepherd_books
Uncontrolled peeking during an A/B test is a serious problem, but there are multiple ways to properly allow peeking in support the most important use case: early aborting of negative tests.
See summary note at https://t.co/pkeGGIWtTm
#abtest #experimentguide #ABTestingAdvancedTopics
What is the Overall Evaluation Criterion for the $400M Golden Gate bridge suicide nets project?
https://t.co/FVqT4zbmPd
#OEC #experimentguide
How do p-values vary over time? What happens to p-values when you test for a sample ratio mismatch (SRM) and actually have one?
Short doc with simulations at https://t.co/U1O3D7vc4z
#abtesting #experimentguide #SRM #pvalues
Is NPS (Net Promoter Score) a Useful Target Metric in A/B tests?
Short doc answering this at https://t.co/pUHQku5QdA
Thanks a lot to @ItamarGilad, @mgershoff, @lukasvermeer, @JeffSauro, Jeff Sauro @MeasuringU, @georgizgeorgiev
#abtest #NPS #experimentguide
Interesting question by guest speaker @mgershoff in my A/B Testing class yesterday (https://t.co/mmGUJ97rYh).
Is A/B testing a causal driver of better company performance, or is it confounded with employees' ability?
#abtesting #experimentguide #bayesian #frequentist #startup
My slides for A/B Testing Intuition Busters at #KDD2022 are at https://t.co/5UeIZJ5Kij
Next Monday I start teaching the 10-hour interactive Zoom class with @get_sphere: https://t.co/mmGUJ96U8J
@alexdeng @lukasvermeer
#experimentguide #abtest
If you think large companies with a massive userbase (Amazon, Google) have an easy time detecting tiny changes in #abtests, you’re wrong!
The number of users needed grows quadratically in the sensitivity, so such companies cannot detect changes of $10M/year.
#experimentguide
The paper "A/B Testing Intuition Busters: Common Misunderstandings in Online Controlled Experiments" with my co-authors @alexdeng and @lukasvermeer will appear in KDD 2022.
See https://t.co/ixgLhxdp4T
#abtest #intuitionbusters #experimentguide
My #flagship22 keynote is now available to view at https://t.co/Pz95L8orQQ . A/B testing examples, feature flags, culture.
The 10-hour interactive Zoom class by @scholarsite starts March 28: https://t.co/EKvFsVIXwU
#flagship #abtesting #experimentguide #flagship #hippo
Be careful with computing p-values to 6 digits of precision from unreliable data that supposedly confirm fuzzy hypotheses by a HiPPO
#abtesting #pvalue #hippo #experimentguide
I am teaching a live Zoom class with @scholar_site: Accelerating Innovation with A/B Testing: https://t.co/EKvFsVIXwU
The class takes place Jan 31-Feb 10, 2022 (5 x 2hr sessions) at 8AM PST.
#ABtesting #ExperimentGuide #experimentation
Last Seen Hashtags on Sotwe
Naagin7
Seen from Australia
public
Seen from United States
حصريات_كـلـوز
Seen from Pakistan
Hoyes
Seen from Japan
porno
Seen from Thailand
รับงานหนามแดง
Seen from Thailand
baited
Seen from United Kingdom
gaybektim
Seen from Indonesia
คาชุดนักเรียน
Seen from Thailand
CampecheVibra
Seen from United States
Trends for you
Most Popular Users
Elon Musk 
@elonmusk
240.2M followers
Barack Obama
@barackobama
119.3M followers
Donald J. Trump
@realdonaldtrump
111.6M followers
Cristiano Ronaldo
@cristiano
109.4M followers
Narendra Modi
@narendramodi
106.9M followers
Rihanna
@rihanna
97.4M followers
NASA
@nasa
92.1M followers
Justin Bieber
@justinbieber
90.7M followers
KATY PERRY
@katyperry
87.1M followers
Taylor Swift
@taylorswift13
80.9M followers
Lady Gaga
@ladygaga
72.4M followers
Kim Kardashian
@kimkardashian
69.5M followers
Virat Kohli
@imvkohli
68.9M followers
YouTube
@youtube
68.6M followers
Bill Gates
@billgates
63.5M followers
The Ellen Show
@theellenshow
62.5M followers
CNN
@cnn
61.9M followers
Neymar Jr
@neymarjr
61.6M followers
X
@x
60.9M followers
Selena Gomez
@selenagomez
60.2M followers