Incrementality Testing for Google AdsHow to Actually Measure It
By Santosh Kumar, Founder of Shopiator
Platform-reported ROAS overstates real impact. Here's how to run a real incrementality test on Google Ads and read the results without fooling yourself.
Incrementality testing measures the sales you'd lose if a campaign turned off, not the sales the platform attributes to it. The two numbers are always different, and for brand search and retargeting they can be wildly different, because platform attribution counts conversions that would have happened anyway. Only a controlled experiment isolates the real lift.
The standard method is a geo holdout: split matched regions into test and control groups, run ads in test regions only, and compare actual revenue lift against what the platform claims. This isn't a one-time exercise. Spend levels, seasonality, and competitive pressure change, so the incremental value of a campaign changes with them.
This post covers how to structure a geo holdout test, how long to run it, and which Google Ads campaign types are worth testing first because their reported-vs-real gap tends to be largest.
Why Platform-Reported ROAS Isn't Real ROAS
Google Ads attributes a conversion to the last (or a modeled) ad click before purchase, regardless of whether that click actually changed the buyer's decision. A customer who was already going to buy and searched your brand name gets counted the same as a customer who discovered you cold through a prospecting campaign. Both show up as "ROAS," but only one represents money the ad spend actually generated.
This gap is largest on brand search and retargeting/remarketing campaigns, where the audience already has high purchase intent independent of the ad. It's smallest on cold prospecting campaigns like non-brand search or PMax targeting new audiences, where the ad is more likely to be the actual reason for the sale.
Even reported ROAS alone, before incrementality enters the picture, moves more than most advertisers assume. Across the same channel and the same time period, Performance Max averaged 4.57x ROAS in the US, 3.42x in the UK, 2.64x in Australia, and 2.51x in Canada, all under identical platform attribution. That's roughly a 2x spread on the attributed number by market alone, which is a preview of how much further that figure can move once you strip out conversions that would have happened anyway.
Setting Up a Geo Holdout Test
Split your served geographies into two matched groups based on historical revenue similarity, not just population size. Pause the campaign you're testing in the control group's geos while running it as normal in the test group's geos. Compare actual revenue (not platform-attributed conversions) between the two groups over the test period, adjusted for any baseline difference recorded before the test started.
Four weeks is the practical minimum for most DTC accounts to get past week-to-week noise, and six to eight weeks is better for lower-volume accounts or highly seasonal products. Running a test for less than two weeks produces numbers you can't trust either way.
Worked example: reading a geo holdout result
- ▶Illustrative numbers, not a real client's figures. A brand search campaign is tested across 6 weeks with matched geo groups.
- ▶Test-group geos (campaign running): $184,000 in actual Shopify revenue.
- ▶Control-group geos (campaign paused): $158,000 over the same window.
- ▶Lift: $26,000, the incremental revenue attributable to the campaign, not the full $184,000.
- ▶Google Ads reports $172,000 in conversion value from that campaign at a platform-attributed 400% ROAS on $43,000 spend.
- ▶Real incremental ROAS: $26,000 / $43,000, roughly 60%, a fraction of the reported number, because most of that $184,000 in test-group revenue would have happened anyway from existing brand demand.
Which Campaigns to Test First
Test brand search and remarketing campaigns before non-brand or prospecting campaigns. These are the campaigns most likely to be overstated by platform attribution, and they're usually the ones with budget increases sitting on the table based on inflated ROAS numbers. If a geo holdout shows brand search has minimal incremental lift, that's the highest-leverage finding available, because it directly informs a rule that should already be standard practice: brand campaigns run on Maximize Clicks with a capped CPC, never on target ROAS, because there's no reliable incremental signal to optimize against in the first place.
| Campaign type | Typically observed reported-vs-real ROAS gap | Test priority |
|---|---|---|
| Brand search | Reported ROAS often 2-4x higher than incremental ROAS | Test first, highest budget-decision impact |
| Remarketing/retargeting | Reported ROAS often 1.5-3x higher than incremental | Test second, especially on aggressive frequency caps |
| PMax (mixed brand + non-brand) | Reported ROAS often 1.3-2x higher, driven by brand bleed | Test after brand exclusion lists are confirmed working |
| Non-brand search | Reported and incremental ROAS usually close, smaller gap | Lower priority, mainly useful as a sanity-check baseline |
| Cold prospecting PMax/Display | Smallest gap of any campaign type tested | Lowest priority, spend is more likely genuinely incremental |
Illustrative ranges based on typical geo holdout patterns, not a specific account's results. Actual gaps vary by brand strength and category.
Reading Results Without Fooling Yourself
A geo holdout that shows 40% incrementality on a campaign platform-reports as 400% ROAS doesn't mean the campaign is bad. It means the campaign's real ROAS is closer to 160%, and budget decisions should be made against that number, not the inflated one. Shopiator is a Google Ads agency for ecommerce brands, run personally by a 16-year PPC specialist with expertise in DTC, supplement/healthcare compliance, PMax architecture, and incrementality testing, and the recurring lesson across accounts is that clients who never ran an incrementality test are almost always over-crediting their brand and retargeting spend.
When Incrementality Testing Isn't Worth the Overhead
Below roughly $10-15k/month in Google Ads spend, a geo holdout often doesn't have enough volume per geo group to produce a statistically usable result, and the operational cost of running one outweighs the insight. At that spend level, a simpler heuristic works: assume brand and remarketing ROAS is overstated by 40-60% and budget accordingly, then revisit real testing once spend scales.
Do / Don't
- ▶Do: match geo groups by historical revenue similarity before splitting into test and control, not just by population or region count.
- ▶Don't: compare before-and-after periods on the same geos instead of a true control group; that conflates the test with seasonality.
- ▶Do: run the test at least 4 weeks, 6-8 for lower-volume or seasonal products, before drawing conclusions.
- ▶Don't: trust a geo holdout under 2 weeks; normal week-to-week revenue noise will swamp the result either direction.
- ▶Do: test brand search and remarketing first since they typically carry the largest reported-vs-real gap and the most budget leverage.
- ▶Don't: run a formal geo holdout below roughly $10-15k/month in spend; use the 40-60% discount heuristic instead until volume justifies it.
Frequently Asked Questions
What's the difference between attributed ROAS and incremental ROAS?
Attributed ROAS is what Google Ads reports based on its attribution model, which counts conversions that would have happened without the ad. Incremental ROAS is the actual lift measured by a controlled test, and it's almost always lower, sometimes dramatically so on brand and retargeting campaigns.
How long should a geo holdout test run?
Four weeks minimum for most accounts, six to eight weeks for lower-volume or seasonal products. Anything under two weeks gets swamped by normal week-to-week revenue noise and won't give a reliable answer.
Which campaign types have the biggest gap between reported and real ROAS?
Brand search and remarketing/retargeting campaigns, because they target audiences with existing purchase intent that platform attribution can't distinguish from ad-driven intent. Cold prospecting campaigns like non-brand search tend to have a smaller gap.
Is incrementality testing worth it for small accounts?
Below roughly $10-15k/month in spend, a geo holdout usually lacks the volume for a statistically reliable result. A practical workaround is discounting brand and remarketing ROAS by 40-60% as a default assumption until spend justifies real testing.
Does incrementality testing replace platform reporting entirely?
No. Platform reporting is still useful for day-to-day optimization signals like which keywords or products are converting. Incrementality testing answers a different question: how much of that reported performance is actually caused by the ad spend, which matters most for budget allocation decisions.
Can incrementality testing be done without pausing ads anywhere?
Not reliably. A geo holdout requires a true control group where the campaign doesn't run, because comparing before-and-after periods on the same geos conflates the test with seasonality and other changes. Some platforms offer conversion lift studies as an alternative, but they're generally less rigorous than a self-run geo holdout.
Want the operator to check this on your account?
The Operator Audit is a full manual review of your account: 10 recommendation sets, a live findings call and a prioritized action plan. The $1,299 comes off your first month if you hire us within 30 days.
Keep reading
Supplement and Healthcare Ecommerce Compliance in Google Ads
Supplement and healthcare advertisers get suspended over specific, predictable policy triggers. Here's what they are and how to advertise without tripping them.
ReadGoogle AdsFeeder vs Converter Search Campaigns: Why You Need Both
A feeder campaign on exact match Maximize Clicks and a converter campaign on broad match tROAS solve different problems. Running only one leaves performance on the table.
ReadGoogle AdsBrand Bidding for Ecommerce: Why Brand Never Runs on tROAS
Target ROAS and smart bidding on brand search campaigns waste budget on customers who were already buying. Here's the bid strategy that actually fits brand traffic.
Read