To figure out what’s really working, you’re hearing two different philosophies. The first is about getting to unimpeachable, causal truth through rigorous experiments. The second is about getting directionally correct answers fast enough to make daily decisions without breaking the bank. Both have a point.
Camp A: Experimental Testing is the Only Truth
This camp argues that since on-platform metrics from Meta and Google are inherently flawed and biased, the only way to measure true lift is with a controlled experiment. You can’t trust what the platforms tell you because they are grading their own homework. Olivia Kory from Haus made this point on a Limited Supply episode, advocating for geotesting (or geolift testing) as the gold standard. In this method, you split a geographic area into a test group that sees your ads and a control group that doesn't. The difference in sales between the two regions is your true incremental lift.
Adherents of this view don't talk about ROAS; they talk about iROAS (incremental ROAS). As discussed on multiple Ecommerce Playbook episodes, this is the core metric you get from running formal scale or holdout tests, often with the help of a third-party platform like Measured. This approach strips away all the vanity metrics and correlations to isolate causality. It’s the closest you can get to scientific proof that your ad dollars are making a difference. The downside? These tests can be expensive and slow, giving you a perfect answer to a question you had three months ago.
Camp B: Modeled Data Gets You Close Enough
This second school of thought prioritizes speed and practicality. The argument is that while a perfect geolift test is great, you can’t run one every day. For daily budget allocation, you need a more dynamic signal. Taylor Holliday often talks about this on Ecommerce Playbook, advocating for the use of an “incrementality factor” (or i-factor). This is a multiplier you apply to your platform-reported results to bring them closer to reality.
This factor might come from a past geolift test, industry benchmarks, or a media mix model. The point is to create a working model that's “good enough” for decision-making. For example, you might discover your Facebook campaigns have a 1.5x incrementality factor, meaning for every dollar Meta reports, you're actually generating $1.50 in true incremental revenue. Richard Gaffin and Taylor Holiday also discussed how this plays out in a multi-channel context, noting that paid social often creates a significant incremental lift on Amazon sales. By understanding this halo effect, a campaign that looks break-even on Meta might actually be very profitable for the business overall.
I believe the most effective strategy is a hybrid of these two camps. You can’t manage what you don’t measure, but you also can’t let the pursuit of perfection paralyze you. The reality is that Camp A and Camp B need each other. Running a quarterly geolift test (Camp A) gives you a highly accurate iROAS read. You can then take that result and use it as your “i-factor” to power a more dynamic, daily modeled approach (Camp B). This way, your model isn’t based on guesswork; it’s calibrated against reality. As described in an Ecommerce Playbook recap, you can use that validated incrementality factor to adjust the ROAS or CPA targets you set in-platform, effectively translating a high-level strategic insight into day-to-day tactical execution.
Your right first move depends on your scale. If you're a smaller brand, you likely can’t afford a six-figure incrementality study. Start with Camp B's principles. Watch your Marketing Efficiency Ratio (MER) like a hawk. When you increase ad spend, does your total revenue go up by a profitable amount? Look for those halo effects on other channels. If you’re a larger brand spending significant money on paid channels, you have to invest in formal Incrementality (Measurement). Running quarterly tests to validate your models isn't a luxury; it’s a core part of responsible Marketing Measurement.