Big-Swing vs. Incremental: Our Ecommerce CRO Testing Framework
Marketing Operators · with Connor MacDonald, Cody Plofker, Connor Rolain · July 7, 2026 · 61 min
Summary
This episode dives into the critical decision-making process for Conversion Rate Optimization (CRO) in ecommerce, exploring when to implement radical site overhauls versus continuous incremental improvements. The hosts and guests share real-world examples from their brands, HexClad, Ridge, and Jones Road Beauty, demonstrating how their testing frameworks lead to significant revenue per user (RPU) lifts and sustained growth. This is a must-listen for ecommerce operators looking to refine their CRO strategies and understand the nuances of A/B testing for maximum impact.
Key takeaways
For significant lifts, consider 'big-swing' CRO tests like a complete theme rebuild, but understand the potential risks and resources required.
Incremental CRO wins can accumulate quickly; for example, HexClad saw a 27% RPU lift from a single collection-page interstitial.
Jones Road Beauty's success with six consecutive winning tests highlights the importance of qualitative research to inform test hypotheses.
Running more than two variants in A/B tests can accelerate learning, provided you have sufficient traffic and statistical rigor.
Develop a holistic testing framework that balances rapid iteration with thorough analysis to avoid
Deploy a qualitative research stack to inform test hypotheses and increase the likelihood of winning tests.
“All bad tests turn good and all good tests turn bad.” How do you run ecommerce A/B testing faster without sacrificing statistical confidence? Connor MacDonald (CMO, Ridge), Cody Plofker (CEO, Jones Road Beauty), and Connor Rolain (Head of Growth, HexClad) debate when to burn the current site down and rebuild versus grinding out incremental wins one test at a time. The answer depends on what you can afford to get wrong. They cover Ridge’s live theme rebuild, HexClad’s 27% RPU lift from a single collection-page interstitial in Japan, and the case for running four variants instead of two. Cody also breaks down the qualitative research stack that helped Jones Road Beauty string together six consecutive winning tests. Powered By Motion Creative Benchmarks 2026 https://motionapp.com/thumbstop-pulse/creative-benchmarks-2026?utm_campaign=marketing-operators&utm_medium=sponsor&utm_content=creative-benchmarks-2026&utm_source=marketing-operators-podcast Haushttps://www.haus.io/operators Richpanelhttps://9ops.co/richpanel Aftersellhttps://9ops.co/aftersell-mops Operators Newsletterhttps://9operators.com/
