The most significant change to A/B testing recently isn't a new methodology, but how platform AI, especially on Meta, has reshaped the fundamentals. A few years ago, the best practice was to build complex frameworks with dozens of hyper-specific audiences and bid strategies. Today, that approach is obsolete. The rigor has shifted away from manually testing audiences and toward systematically testing creative. This is because Meta's algorithm is now far better at finding the right customer for an ad than any human media buyer is at defining them through interest and demographic targeting.
So, what does that mean for your testing framework? First, you can stop obsessing over audience segmentation in your testing campaigns. As Andrew Faris explains on The Andrew Faris Podcast, broad audiences paired with a diverse set of creatives consistently outperform granular targeting. The algorithm needs variety. When you feed it different creative concepts, you allow it to find pockets of performance across a wide audience that you would have never discovered with narrow targeting. Your job is no longer to tell Meta exactly who to target, but to provide the creative fuel it needs to find those people on its own.
The core of a modern testing framework is a high-velocity, structured approach to creative development. The team at Pilothouse, who show up often on the DTC Podcast, have a great system for this. They advocate for dedicating a small portion of the budget, around 5-10%, to a specific creative testing campaign. This campaign uses Ad Set Budget Optimization (ABO) to ensure each new creative gets a fair shot with a controlled budget. You're not trying to achieve your target CPA here; you're looking for directional signals like high click-through rates or low cost-per-hook.
Inside this testing campaign, the key is to be methodical. As Marco Grishaber suggests on the Ecommerce Exits Podcast, you should practice incremental testing by changing only one variable at a time. You can test a core "angle"—for example, whether customers respond more to a "convenience" angle or a "luxury" angle. Within that winning angle, you can then test variations of the hook, the visual style, or the call-to-action. Barry Hott made a great point on Ecommerce Conversations that raw, emotional, or even "ugly" ads can be incredibly effective. Don't be afraid to test creative that breaks from your brand's polished aesthetic. The goal is to build a library of winning components and angles.
Once a new creative proves itself in the testing campaign, it gets "graduated" to your main scaling campaign. This is typically a simplified setup using Campaign Budget Optimization (CBO) or Advantage+ with broad targeting. This is where you let Meta's algorithm do its work, spending the budget on the best-performing ads and finding the right audience segments for each one. Your Facebook Ads effectiveness will become much more stable.
Even bid strategy testing has been simplified. On an episode of Ecommerce Playbook, Taylor Cain breaks down how to use cost caps, but it's presented as a tool for specific situations, not the default setting. For most advertisers, letting Advantage+ manage bids to achieve the lowest cost per conversion is the most effective strategy. Your bidding "test" is simpler now: run a scaling campaign with the default lowest-cost bidding and see if the CPA is sustainable. If not, you can test a cost cap campaign to see if you can acquire customers more efficiently, knowing it may come at the cost of volume.
Ultimately, a rigorous framework that works right now is one that embraces the platform's automation. Simon Wool of Little Spoon summed it up well on the DTC Podcast when he talked about the importance of a data-driven creative strategy. The rigor is in your creative process and analysis, not in manually managing countless ad sets. You feed the machine, analyze what it learns, and send it better food next time.