Ty Cohen

[Ty Cohen @tymyshoe86]

Cody is making full landing pages, so then I would assume a full 'redirect' test. These are often the toughest because even Intelligems resorts to a javascript redirect which can often result in a flash of the old page.

There was a recent debate around apps like Intelligems hurting ad performance because it confuses the algo + adds UX issues like speed issues and flashes of another page before redirect to the user. https://t.co/1P1Cr3Bz7h

I was curious if @codyplof felt AB testing in the ad platforms was a safer bet.

[Connor Martin @connor_m]

Nobodies ready for the conversation that A/B Testing apps cause a massive negative impact for paid traffic spend.

An extremely large benefit of having multiple brands at scale is the ability to analyse the impact of otherwise 'harmless' site or funnel changes.

Below is our NCPA, anytime we have a AB test on rCPMs go up at least 20% OR NCPA goes up 10-30%, or in the below example, as much as 80% increase in NCPA.
(One of the below tests was a 'winner' with 30% increase in PPS)

Take a guess when AB tests are live or stopped, each small climb in NCPA is when a AB test starts, the sharp reduction is when a AB test is stopped, or in today's case, when AB testing apps are completely uninstalled.

"Oh that's simply because it's a failed AB Test"
Wrong.

AB Testing is inherently flawed from the beginning for DTC brands, as your PPS or RPS is the main tracked net gain.

However this isn't the goal for DTC brands, as your paid traffic makes up over 50% of your traffic.

The main itemised metrics for growth for DTC brands are NCPA, CPMs, Account Center costs, etc etc.

Next time you run a large AB Test, track these over PPS and RPS. What you'll see is when you accompany the 'increase' in RPS on site, you'll 9/10 times see an unnormally high increase in above metrics.

Or, simply run a double fresh URL AB test directly through Meta, the AB test that you thought was 50%+ in RPS, will end up failing in Meta. The reverse happens too, that AB test you thought would be a massive winner but failed? Could win via Meta directly.

This isn't a direct attack on AB Testing or CRO agencies, and isn't something to ridicule me for below, our sites get over 7m human sessions a month, I have vigorously tested this, I clearly WANT AB testing to work, they are my stores, we're bootstrapped, wins from an AB test literally go into my pocket.

CRO agencies would see a massive growth actually agreeing with this, and working with brands to test website changes the way they should be tested, directly through ad accounts, and acknowledging that most agencies do not know how to calculate RPS or PPS, which should include CACs.

Other things you can do to confirm this isn't BS -

- After a winning AB Test, set the test live on site, and AB test it again against the 'old' site.

- Run a test where the A and B are both the exact same site.

- Try running a three-way test, A (original), B (test), C (original), I can guarantee you, A and C do not end up the same after significance.

This is not app related, we've used all of them.

In fact, the next app to enable AB Testing via ad accounts, i'll use it, great app idea, i'll fund it for whoever wants to create this.

[Ty Cohen @tymyshoe86]

@typeangled Will this make it into the next patch version? Minor version?

[Ty Cohen @tymyshoe86]

@codyplof I'm building this out right now too. Have you developed a style guide.liquid page for claude code to refer to? I developed a set of 30 sections for it to better understand our design system and aesthetic.

Next is making sure you have custom well defined sections that it can work with. The more concrete .. serving a single
Limited purpose the easier of a job the AI will do at staying within your parameters. Invest heavily into these narrowly focused sections. Put the time and effort in here. Wildly complicated theme sections with nested block options will choke the AI

Develop a skill file that maps out all of these sections and let it know what kind of structure is defined - it'll be easier to know when to choose this type of block.

Next step built a custom MCP server that has knowledge of your image files. This is the hardest thing to get your pages looking good. How do you get images in? I built an AI to set great Alt text on every image in my repo. Then MCP server to show all images and let the AI decide which image to choose (even if you get a better temporary placeholder here, it’s better then no images)

Of course keep on iterating on the skill file to get better results.

[Ty Cohen @tymyshoe86]

@AnthropicAI you have a significant bug in your Console billing system.

$200 in credits and I can't get out of the 'Evaluation Tier'.

System is marking my purchases as a 'grant', not a purchase.

Have contacted support multiple times with no response. Can someone help?

[Ty Cohen @tymyshoe86]

@ricsdeol @gnoshme @davetron5000 @heroku Having the same issue.

[Ty Cohen @tymyshoe86]

@EcomWorldConf Your site is not working. Can't view any talks.

[Ty Cohen @tymyshoe86]

@GovMurphy @FirstLadyNJ Please allow pregnant moms to have Doulas in addition to their spouse. Moms could really use more support right now! Thanks!

[Archive: Governor Andrew Cuomo @NYGovCuomo]

Holding a briefing with updates on #Coronavirus. WATCH LIVE: https://t.co/gLjk74Ghtt

[Governor Phil Murphy @GovMurphy]

Holding a COVID-19 briefing. WATCH: https://t.co/kRquJyEITL

[Ty Cohen @tymyshoe86]

@IntuitDev @IntuitDevStatus I'm experiencing issues with the QBO API. Specifically getting back empty data on requests. There are several other errors happening too.

[Ty Cohen @tymyshoe86]

@tomfriedman Love your books. Anything new on the horizon?

[Ty Cohen @tymyshoe86]

@mandrillapp @Mailchimp We're also having issues with Emails being dropped / emails not being sent.

[Ty Cohen @tymyshoe86]

@vegancomics @PrintNode Yes, @PrintNode . We're also having trouble.

[Ty Cohen @tymyshoe86]

@PrintNode Yes, we are experiencing connectivity issues. Please let us know when the issue is resolved.

[Ty Cohen @tymyshoe86]

@PrintNode @PrintNode Please let us know when this is resolved.

[Good Kyle @KyleMcDowell86]

"they're okay i guess"

-Tony the Tiger when he's off his meds

[Mike Birbiglia @birbigs]

I'm into Indian rock. Or....um...indie rock. You know what I mean. I'm cool.