Juan Cruz Giusto

Head of eCom @ Mars Men | Scaling subscription brands through experimentation and finance

[Juan Cruz Giusto @JCGiusto]

Want to grow your DTC Subscription brand?⁣
⁣
Learn 5 powerful strategies 👇⁣
⁣
𝟭. 𝗖𝗿𝗲𝗮𝘁𝗲 𝗮𝗻 𝗜𝗿𝗿𝗲𝘀𝗶𝘀𝘁𝗶𝗯𝗹𝗲 𝗦𝘂𝗯𝘀𝗰𝗿𝗶𝗽𝘁𝗶𝗼𝗻 𝗢𝗳𝗳𝗲𝗿⁣
⁣
People who purchased one time are unlikely to subscribe on their second or third order.⁣
⁣
That's why you need to focus on making the subscription option a no-brainer. Think of discounts, free gifts, or access to exclusive resources to increase the value of that option.⁣

𝟮. 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗶𝗺𝗽𝗿𝗼𝘃𝗶𝗻𝗴 𝘁𝗵𝗲 𝟮𝗻𝗱 𝗮𝗻𝗱 𝟯𝗿𝗱 𝗼𝗿𝗱𝗲𝗿 𝗿𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗿𝗮𝘁𝗲⁣
⁣
Most subscription churn happens in the 2nd and 3rd order, which is actually where you can have the biggest impact.⁣
⁣
Leverage the reciprocity bias and surprise your customers with unexpected gifts and resources to keep them hooked.⁣

𝟯. 𝗦𝗲𝗹𝗹 𝘁𝗵𝗲 𝗹𝗼𝗻𝗴-𝘁𝗲𝗿𝗺 𝘁𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻⁣

How long your subscribers stay with you is a function of how you position your product.⁣

Sell the long-term transformation and not empty short-term promises. Help users build a habit around your product and let them know what they can expect 1, 2, 6, and 12 months from now.⁣

𝟰. 𝗞𝗻𝗼𝘄 𝘆𝗼𝘂𝗿 𝗻𝘂𝗺𝗯𝗲𝗿𝘀⁣
⁣
Don't assume subscribers are better than one-time purchasers.⁣
⁣
Subs tend to have worse gross margins since you are giving them free gifts, discounts, free shipping, and more.⁣
⁣
Increasing your number of subscribers is, at the end of the day, a proxy for actual long-term profits.⁣

𝟱. 𝗕𝘂𝗶𝗹𝗱 𝗮𝗻 𝗲𝗰𝗼𝘀𝘆𝘀𝘁𝗲𝗺 𝗮𝗿𝗼𝘂𝗻𝗱 𝘆𝗼𝘂𝗿 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗽𝘁𝗶𝗼𝗻⁣
⁣
Your "product" doesn't need to be just your product. Build a community around your vision, create resources that help your customers achieve their goals faster, partner with complementary businesses.⁣
⁣
Whatever you create will increase both conversion and retention rates. ⁣

[Juan Cruz Giusto @JCGiusto]

I see the same CRO tests being regurgitated again and again.

No creativity, no research behind it, no strategy informing it.

Just go through the best-practice list that everyone uses.

That's not how you make a difference and take your business to the next level.

The companies that win will be the ones using experimentation to solve a real strategic problem, improve the user experience, and revolutionize their business model.

If not, someone else is going to do it before you.

[Juan Cruz Giusto @JCGiusto]

Returns are the silent margin killer 💀

Invisible in platform reports, but devastating on the real P&L.

I've worked with brands that have up to 20% return rates.

A 5% reduction in returns is equivalent to an 8-12% lift in net margin.

The action plan:

→ Run questionnaires to understand the reasons
→ Study the responses and build a specific plan
→ Improve product quality
→ Implement AI-powered size guides
→ Add clear instruction manuals

The risk of implementing these changes is almost zero, and the impact on your P&L can be brutal.

While others chase more traffic, you optimize where the bottleneck actually is.

[Juan Cruz Giusto @JCGiusto]

Your CFO hates discounts (and he's right)

Here's why. And what your eCommerce can do about it.

--

Plenty of studies show that a big chunk of discount-driven sales aren't incremental. They generate sales that would have happened anyway.

In his book "How Brands Grow", Byron Sharp says:

"What happens in practice is that brands are routinely discounted below their normal price to generate a short-term sales lift. (...)

In many cases, they don't generate additional profit, because the margin sacrificed on those sales would have been earned anyway at full price."

Takeaway: Don't lean on them to hit revenue targets.

Push your team to think of more creative ways to promote the brand.

You'll see the ideas that show up once the "easy" lever is off the table.

Use discounts sparingly to:

→ Increase their impact
→ Protect margins
→ Protect the brand

The discount should be the exception, not the rule.

And when that's the case, the ideas that actually move the needle on profit start to appear.

[Juan Cruz Giusto @JCGiusto]

Retention rates might work for SaaS that charge the same amount every month.

But for eCommerce brands, they don't tell the full story of what's happening.

Let me explain what you should use instead to increase your customer's LTV.

At the surface, it might look like SaaS and eCommerce are similar.

But when you dig deeper, they are very different:

⚫ First, eCom tends to have different AOVs based on the customer type (subs vs one-time).

⚫ Second, they have costs associated with selling the product.

And even though they are critical to assess your eCom's health, this information is completely missing from usual retention rates and cohorts.

Think about it...

If you compare users that subscribed vs those who don't, you might see way higher retention rates on the former.

But what if those subscribers have lower AOV, and since we are offering discounts and free gifts, we are only getting our money back 3 months later?

Sure, retention rates look great, but our short-term profits are not.

What's troublesome is that retention rates are the default graph on Shopify Analytics.

So, what can you do instead?

Start tracking LTV Progression, which considers retention rates as well as AOV.

The good thing is that it's very easy to check on Shopify.

Just go to the Customer Cohort Analysis and, on the right, change the metric to Amount Spent per Customer.

The graph will give you a clear picture of how a Cohort's Average LTV progresses month to month.

And you can even filter it by Subscriber or not!

Unfortunately, it doesn't let you discount COGS or Shipping costs yet, but it's something you can calculate if you know your CAC, COGS & Shipping costs.

I'm not saying that retention curves are useless - quite the opposite.

They are a great tool to understand when most users stop buying from you, canceling your subscriptions, or when the retention rate stabilizes.

Analyze both graphs, this is how you'll get the most insights.

[Juan Cruz Giusto @JCGiusto]

Having good ideas is only a small part of CRO and Experimentation.

Without a SYSTEM with 3 specific components, your program will fail.

Let me explain…

I've been spending time on Twitter and LinkedIn

(more than I should have, really)

And have seen dozens of people showing their "successful" tests and amazing ideas to try out.

From changing quantity selectors to leveraging social proof and more.

And that made me realize that experiment ideas are becoming a commodity.

What differentiates a successful Experimentation Program from a mediocre one is A SYSTEM.

And this system needs to have 3 specific components:

1. A research methodology to uncover your users' specific needs and a prioritization framework to tackle the biggest ideas

2. Statistical procedures to ensure that winners are REAL WINNERS that translate into ACTUAL profits

3. A decentralized platform where people can learn from past experiments, upload their ideas (and ideally test them themselves).

Without them, you are just throwing spaghetti at the wall and hoping something sticks.
Actually, it's even worse. Your spaghetti is not nearly cooked.

Conclusion?

Build a system where ideas can flourish.

Because ideas are everywhere.

But effective systems are rare gems.

[Juan Cruz Giusto @JCGiusto]

Traditional CRO fails subscription businesses.

Here's why:

You run a test pushing subscriptions and see:

• Conversion Rate: +1.25%
• AOV: -7%
• ARPU: -5.75%

Standard CRO logic says to kill this test immediately.

But track those cohorts for 60 days, and suddenly:

Variant LTV: +7% higher than control!!!

That's the subscription flywheel at work.

But doing it right is hard. You'll need:

• A/B test integration with your ecom platform
• Customer cohort tagging
• 30/60/90-day LTV tracking
• Proper statistical methods

[Juan Cruz Giusto @JCGiusto]

Here's why Conversion Rate is a shitty metric

And what you should focus on instead...

In the world of eCommerce and Conversion Rate Optimization (CRO), we often celebrate high conversion rates. However, I believe this can be a very misleading metric.

Why?

Because it doesn't tell the whole story of what's happening.

You can increase conversion rates by introducing discount codes, giving gifts, and lowering the free shipping threshold. But these actions can also lower the Average Order Value (AOV) more than the increase in conversion rate, leading to a lower Average Revenue Per User (ARPU).

In other words, you might increase the conversion rate and conclude the experiment as positive, but end up with less money in the bank.

So, what can you do instead?

The first step is to start tracking ARPU. This way, you take into account both conversion rates and AOV.

But there's an even deeper level, and I believe it’s where we are heading.

The next step in eCom growth and CRO should focus on profits, not revenue.

Profits consider revenue minus Cost of Goods Sold (COGS), shipping, and ideally, average return rates per product. In this way, test results get closer to what really matters: cash in the bank.

Implementing this profit-focused approach, though, is complicated. It requires an A/B testing tool that connects with your backend to consider COGS, shipping, and returns.

The only app I’ve seen that has these features implemented is Intelligems within the Shopify ecosystem. But it’s only a matter of time until all A/B testing platforms move in that direction.

I truly believe this is the future of CRO and eCom growth: focusing on profit, rather than a binary metric like conversion rate.

What’s your take?

[Juan Cruz Giusto @JCGiusto]

A/B tests 𝗮𝗿𝗲 𝗳𝗮𝗿 𝗺𝗼𝗿𝗲 𝗽𝗼𝘄𝗲𝗿𝗳𝘂𝗹 than I initially believed.⁣
⁣
Yes, they help you reduce risk, but that's not all...⁣

𝗧𝗵𝗲 𝘁𝗿𝘂𝗲 𝗽𝗼𝘄𝗲𝗿 𝗼𝗳 𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗲𝗺𝗲𝗿𝗴𝗲𝘀 𝘄𝗵𝗲𝗻 𝗲𝘅𝗽𝗼𝗻𝗲𝗻𝘁𝗶𝗮𝗹 𝗴𝗿𝗼𝘄𝘁𝗵 𝗸𝗶𝗰𝗸𝘀 𝗶𝗻.⁣
⁣
And that's only possible when you retain the positive effects and discard the negative ones.⁣
⁣
If this discarding process doesn’t occur, the positive and negative impacts end up neutralizing each other.⁣
⁣
The challenge is that it’s not easy to distinguish the experiments that produced uplifts from those that didn’t.⁣
⁣
Unless the impact is obvious, many external factors can cloud the data.⁣
⁣
This is where A/B testing comes in.⁣
⁣
𝗥𝗮𝗻𝗱𝗼𝗺𝗶𝘇𝗲𝗱 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗱 𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝘀 (𝗮𝗸𝗮 𝗔/𝗕 𝘁𝗲𝘀𝘁𝘀) 𝗵𝗲𝗹𝗽 𝘆𝗼𝘂 𝗶𝘀𝗼𝗹𝗮𝘁𝗲 𝗼𝗻𝗲 𝘃𝗮𝗿𝗶𝗮𝗯𝗹e—the change you are introducing—to analyze its impact on the business.⁣
⁣
In this way, you can be (almost) sure if your idea is producing positive or negative results.⁣
⁣
Thus, you keep only the uplifts.⁣
⁣
But why exponential?⁣
⁣
Simple.⁣
⁣
Each positive experiment builds on the previous ones, compounding the effects.⁣
⁣
𝗜𝗻 𝗲𝘀𝘀𝗲𝗻𝗰𝗲, 𝘁𝗵𝗲𝘆 𝗱𝗼𝗻'𝘁 𝗷𝘂𝘀𝘁 𝗮𝗱𝗱 𝘂𝗽; 𝘁𝗵𝗲𝘆 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝘆!⁣
⁣
That's why growth and experimentation programs become more valuable over time.⁣
⁣
Not to mention the insights you gain along the way.⁣
⁣
So, don't shy away from A/B tests—they are an invaluable tool in your growth arsenal.

[Juan Cruz Giusto @JCGiusto]

Your subscription eCommerce North Star Metric can't just be "number of subscribers."

Here's why: If you're only focusing on subscribers, you're completely neglecting one-time buyers - who often represent a big percentage of revenue.

Your ideal NSM needs to consider for BOTH worlds - subscribers AND one-time purchasers, plus AOV, LTV, and (ideally) profit margins.

Is it complex? Yes.

But anything less risks steering your business in the wrong direction.

[Juan Cruz Giusto @JCGiusto]

Here's a counterintuitive truth about subscription businesses: Sometimes, the "losing" tests are actually your biggest winners.

Recently, I ran an experiment that initially looked like a failure:

• Conversion rate: Flat
• AOV: -2%
• Revenue per user: -1.24%

Most would've killed the test.

But here's the plot-twist: subscription take rate increased by 114%.

After 60 days, variant was significantly outperforming control.

Why? Because subscription metrics compound over time.

A seemingly small improvement in subscription rate can deliver exponential returns over months and years.

So, when testing subscription products, don't just look at immediate metrics.

Track cohort performance over time.

Your "failed" test might be your next breakthrough.

[Juan Cruz Giusto @JCGiusto]

Don't tell me that your sms/email/app automation generated 30x ROI...

Unless you've demonstrated incrementality through an A/B test.

Of course they will have an extremely high ROI.

Advertising and the website will do most of the job.

Those apps are only doing the last (and perhaps easiest) part after all the dirty work has been done by others.

And they attribute revenue like they were the only cause.

So next time those apps promise you stupidly high returns of investment, ask them for an A/B test.

It's the only way to really know if they are generating the extra dollars.

In they don't have one, be cautious...

They might be attributing things they aren't actually doing.

If you need any help determining who's who, I'll be happy to give your hand.

[Juan Cruz Giusto @JCGiusto]

There's one key hidden function of A/A Tests that almost nobody in eCom talks about.

But it's key to a CRO & Experimentation Program's success.

And it's this one:

It lets you calculate the value and standard deviation of Average Revenue per User and Profit per User from a specific funnel step.

Sorry, nothing very fancy.

BUT IT'S INCREDIBLY USEFUL so let me explain why.

In order to avoid peeking and increasing your false positive risk, you must do a pre-test analysis.

Meaning, you determine how long your test should run and how many users you need BEFORE you run the test.

For binomial metrics like Conversion Rate, this is pretty easy.

You either get into GA4 or Shopify and calculate it.

But for non-binomial metrics like ARPU or PPU (Profit per User) metrics, it's more complex.

In order to do a pre-test analysis for them, you need their value standard deviation.

And that data is complicated to extract from Shopify or GA4.

Enter A/A Tests.

Apart from being the perfect tool to determine that the experimentation platform is working correctly (no SRM, for example).

It can also help you determine the cohort's ARPU and PPU.

And by downloading the raw data and with a couple of simple calculations, you can arrive at their standard deviation.

So, if you launch an A/A Test on every step of the funnel (Home, PLP, PDP, Cart, and Checkout), you'll get everything you need to perform your pre-test analysis for non-binomial metrics.

More clarity. Better experiment setups. More trustworthy results.

[Juan Cruz Giusto @JCGiusto]

I think you raise some good points but, imo, others have flaws.

In general, it's true that website tests are inherently blind to changes in CPA.

When we run tests, we expect that the increase/decrease in CVR has a direct relationship to CAC, but this relationship is hard to prove.

In theory, increasing CVRs by 10% should decrease CPAs by approx. 10%, but in reality, the effect is hard to isolate and probably not linear - specially if the effect is negative.

Recently I've run an experiment that generated a 10% increase in ARPU (statistically significant, correctly powered) that came through a 25% increase in AOV and a decrease of 15% in CVR.

In theory, it worked. But when we pushed the change to production, CACs, instead of increasing 10%, increased 25%.

My theory is that Meta penalizes your site for converting worse by charging higher CPMs, so you end up getting hit twice. An Intelligems A/B test can never catch that, because that happens before the randomization occurs.

So yeah, observing how CAC/ROAS behave after a big experiment is sent to production is key and, imo, you are right.

Now, let me go through what I disagree with:

- "Anytime we have an AB test on, rCPMs go up at least 20% OR NCPA goes up 10-30%" | Correlation does not equal causation. Proving that launching tests increases your CPA requires way more rigorous analysis and it's most likely not the case. An impact on CPMs might happen if the change introduced is big and there are delays in loading, but after launching hundreds of experiments, it's almost never the case.

- "AB Testing is inherently flawed from the beginning for DTC brands, as your PPS or RPS is the main tracked net gain." | You are throwing the baby out with the bathwater. A/B testing works and it's the best way to measure the impact of changes by isolating them from other variables; we just need to be aware of its inherent limitations.

- "Simply run a double fresh URL AB test directly through Meta, the AB test that you thought was 50%+ in RPS, will end up failing in Meta." | A recent paper shows that Meta/Google are not doing true randomization and their results are not as trustworthy as a correctly run A/B test. You can check that here: https://t.co/fv8wYkUMvH

The cure ends up being worse than the disease.

Also, if your test was underpowered (meaning that it didn't have the sample size needed to catch a true effect), you will most likely not be able to replicate the results - either on a Meta A/B test or an Intelligems test - because they were not true and likely exaggerated from the beginning.

- "After a winning AB test, set the test live on site, and AB test it again against the 'old' site." | If your test was correctly powered, there's a high chance that the results will be similar to the 1st test.

BUT EVEN THEN, there's a 20% chance that the test won't be able to catch the true positive result (assuming you set 80% power for your test) and there's a 10% chance that the positive effect you saw on the 1st test was a false positive (assuming you set a 90% statistical significance threshold).

- "Run a test where the A and B are both the exact same site." | That's called an A/A test and, if you run 10, one of them will most likely be statistically significant. That's normal and expected. If you let it run for longer, it will stabilize and Control will naturally go closer to Variant.

"Try running a three-way test, A (original), B (test), C (original), I can guarantee you, A and C do not end up the same after significance." | Same answer as above.

[Juan Cruz Giusto @JCGiusto]

@iamgalba Did you get a chance to see this @iamgalba ?

[Juan Cruz Giusto @JCGiusto]

@iamgalba Google

[Juan Cruz Giusto @JCGiusto]

@iamgalba Shopping

[Juan Cruz Giusto @JCGiusto]

Contribución Marginal vs Beneficio en eCommerce

#ecommerce #ecom #ecomgrowth #dtc

[Juan Cruz Giusto @JCGiusto]

No Hay Solo Una Métrica Importante en eCommerce

#ecommerce #ecom #ecomgrowth #dtc

[Juan Cruz Giusto @JCGiusto]

La Importancia del aMER en eCommerce

#ecommerce #ecom #ecomgrowth #dtc

[Juan Cruz Giusto @JCGiusto]

¿Qué es la Incrementalidad? Todo eCommerce lo debería entender

#ecommerce #ecom #ecomgrowth #dtc