AnalyticsToolkit

We are passionate about Web Analytics, Statistics & Data-Driven Decisions in Online Marketing.

[AnalyticsToolkit @AnalyticsToolki]

The new standard for state-of-the-art statistical planning and analysis of online A/B tests is here! A game-changing overhaul of Analytics Toolkit brings with it a ton of improvements, new features and functionalities. Full list of changes and updates: https://t.co/r47ObnxMCN

[AnalyticsToolkit @AnalyticsToolki]

Is any of your colleagues skeptical of the value of A/B testing? The business case for experimentation (#abtesting) is explored and showcased through simulations, numbers, and graphs in this article:

👉 https://t.co/Pv3MnGxsZr

#experimentation

[AnalyticsToolkit @AnalyticsToolki]

What if the observed effect in an A/B test is smaller than the minimum detectable effect (MDE) used in planning it? Is it cause for concern? Does it make a test less trustworthy? The answers to the above and more in https://t.co/l3O1wYcUQD

#nhst #statistics #mde

[AnalyticsToolkit @AnalyticsToolki]

Observed power is a tempting concept, but is misused almost by definition. Are you curious about the true power of your statistical test? You're barking the wrong tree of post hoc power. Full article at https://t.co/escYyEBGTX

#statistics #statisticalpower #nhst

[AnalyticsToolkit @AnalyticsToolki]

"Stop AbUsing the Mann-Whitney U Test (MWU)" #abtesting #statistics #DataAnalysis

https://t.co/Dpi5ocgCKS

[AnalyticsToolkit @AnalyticsToolki]

Not sure whether you should adopt sequential A/B tests? Think it's all about testing velocity? Check out this article showcasing the primary value of sequential tests in an accessible manner:
https://t.co/OFcGmbUBgu

[AnalyticsToolkit @AnalyticsToolki]

Worried about the false positive risk of your experimentation program, or a particular test? Can you estimate what proportion of your "winners" are in fact "losers", and does it even matter? This and more #abtesting #statistics in https://t.co/joNLJest7k

[AnalyticsToolkit @AnalyticsToolki]

After more than a decade, we are dropping our Google Analytics API integration and thus all related functionalities. Learn more about why and what you can do to use GA4 A/B test data going forward: https://t.co/AHWegSP6Kx

#googleanalytics #GA4 #abtesting

[AnalyticsToolkit @AnalyticsToolki]

How to run shorter & more efficient A/B tests? This article covers the basics and can lead to 40%+ improvement in the expected test duration. #abtesting #statistics

https://t.co/HoYsU8dmbh

[AnalyticsToolkit @AnalyticsToolki]

A deep discussion on statistical power, minimum detectable effect, and how to design statistical tests in #abtesting:

https://t.co/ny9XtOvrZS

[AnalyticsToolkit @AnalyticsToolki]

What is the best design for a statistical test with sequential evaluation of data at multiple points in time? Should you do group sequential or go full sequential? Is there a time & place for each?

https://t.co/9KYvwsB8Dg

#statistics #abtesting #conversionrateoptimization

[AnalyticsToolkit @AnalyticsToolki]

/2 Seems they are now attempting to add more unrelated queries to suggestions related to our brand name, following a similar pattern. While the damage is not great at this point, the phony suggest is now #2, and if this continues all our suggests will soon be 100% irrelevant.

[AnalyticsToolkit @AnalyticsToolki]

@JohnMu Help needed with weird Google Suggest spam. 100% fake traffic from India brings a completely unrelated query "brandname earn money" up in suggests for "brandname" queries slowly over 4 months. A few real users are now clicking on the phony suggest. What can we do?

[AnalyticsToolkit @AnalyticsToolki]

Session-based metrics see use in online #abtesting, but they can easily lead to erroneous test conclusions. While well-known by some, it seems this issue is underappreciated by others. This article presents the issue in detail:
https://t.co/XgeVxOiodG

[AnalyticsToolkit @AnalyticsToolki]

P-values and confidence intervals: an accessible explanation for practitioners of #cro and #abtesting.

#statistics

https://t.co/Wn4KO22Aio

[AnalyticsToolkit @AnalyticsToolki]

A new glossary entry on A/B testing: https://t.co/4h0MtfY0X7 is now up. Should be a useful resource when introducing someone to online testing and experimentation.

#abtesting #conversionrateoptimization #cro

[AnalyticsToolkit @AnalyticsToolki]

A little update for those using Analytics Toolkit to analyze tests performed in Google Optimize. Now there is the option to import data from tests with a paused/disabled default Control group. Variant 1 has to be assigned the default user experience.

#googleoptimize #abtesting

[AnalyticsToolkit @AnalyticsToolki]

"Statistically rigorous A/B tests are impractical". This and similar beliefs may be hindering your A/B testing program, even if you do not subscribe to them. This article takes on debunking the top misconceptions about scientific rigor in #abtesting :

https://t.co/TWZcyfoFMR

[AnalyticsToolkit @AnalyticsToolki]

@_mightydouche The reseller of our domain apparently had access to make the change in the authoritative DNS servers, so they pointed the records to their own DNS servers (and obviously copied the records there).

[AnalyticsToolkit @AnalyticsToolki]

Our service should be restored shortly for everyone. We were finally able to switch our DNS away from eNom (who have still not solved their issue) and the site should start resolving shortly. Apologies for this quite unprecedented downtime!

[AnalyticsToolkit @AnalyticsToolki]

https://t.co/q4Zg7zEC4k is affected by what appears to be a prolonged unexpected downtime of the eNom nameservers. As the registrar is down as well, this is currently impacting hundreds of thousands of domains with no solution. We expect to be back soon.