Let me introduce myself...
I'm Anthony Morgan CEO and Founder of Enavi.
I went to school for music and stumbled into ecomm.
☝️I know I am not alone in this.
I had the opportunity to start in-house before agency life.
Now I wear too many hats (literally) and mumble about CRO.
@natelagos This is 100% how learnings should be shared through an organization and how the impact of experimentation (ultimately research) compounds rapidly.
We do the same thing at Enavi. So I don’t know if you’d 4x us haha
but regardless I’m big fan of your application of learnings.
@natelagos 100% agree that there is learning to be taken from this.
In my book you have enough confidence to move forward with D over the control.
Also I am all for this approach.
Too often I see learnings from on-site experimentation die at the interpretation of a test result.
@natelagos Meant to say 175.8% lift in RPV not CVR.
Is variant D the right one to go with?
Probably, but you can’t claim $160,812 as the ROI.
If that’s where the 4x the impact comes from, then it’s not actually substantive and trustworthy.
@natelagos This actually proves my point.
Variant D has a +5% to +620% interval. That is incredibly wide and would be wrong to claim this as a 175.8% lift on CVR.
The sample size is too small which is why Variant D 1 out of every 5 times probably won’t be the winner.
@natelagos I’m really not a stats nazi and don’t want to be that guy.
I think you do great work it’s just I’d recommend approaching your testing methodology a little different.
Otherwise your 4x results are going to be full of Type M errors. So not actually 4x the results.
@natelagos If the tool says the lift is somewhere between -2% and +18%, that’s not a winner even if it says 95%. Ship it and you don’t know if you gained 18 or lost 2.
6 variants means 6 intervals, each with a sixth of the traffic. They’re all that wide unless you have a lot of traffic.
@JCGiusto In general showing a single test snapshot does more harm than good.
Good experimentation is more about the cumulative impact solving real business and customer problems has than a single test ran over a 2 weeks.
I’ve seen many “big win” AB Tests not bear the weight of time.
@SarahLevinger Wow. This is what happens when you get enamored with the wrong metrics looking for a sugar rush.
No good long term impact could come from a stupid tactic like this, especially for True Classic.
@ron_ecomm The macro only matters when it’s positive momo… said every chad brand ever
Building trust is the way forward more than ever before. AI is a big reason why. The slop and the way it’s shaping how we shop.
I agree with you - now is a good time to double down on trust!
This is what happens when you bring in a strategist + a research driven experimentation process to new customer acquisition.
Don't settle for average with your current landing pages.
Most performance marketing agencies have no idea what to do with the post-click experience.
This is eerily similar to FERMÀT's claim. Many of you know how firsthand how their agency accelerator fizzled out.
I think you should be spinning up more landing pages, and AI should be supporting, but this is just another solution that at this point is too good to be true...
Page builders are dead.
So we rebuilt our platform for a world that runs on agents.
Introducing Pagedeck: the AI CRO agent that builds high-converting landing pages for every ad.
Let me show you 👇
RT + comment "Pagedeck" and I'll DM you the biggest landing page swipe file
You need a strategist...
One who knows your brand, your customers, and your products.
AI should support in that and ideally build pages.
But it's naive to think AI can do it all.
Who knows maybe it will be better than the low bar your current efforts have set...