Meta Incrementality Testing: Real Case Proves 4x Over-Attribution

Once Upon a time…

A B2C eCommerce that invested 3000€/month in Meta Ads, had the doubt that Meta was attributing more transactions to itself than it actually generated.

In fact, Meta said that it was able to generate customers at an average cost of € 0.54.

This information was almost too good to be true.

As one of the main channels he invested in, he wanted to find out if Meta was telling the truth about his reporting.

How could we find out if Meta was reporting true conversions and measure the real impact of that spend?

Can You Trust Meta Ads Attribution?

How many conversions came from Facebook (Meta) campaigns?

Well, the answer was not Google Analytics.

Google Analytics attribution models only measure how many people bought immediately after clicking on an ad.

"If a person looks at your ad, does not click and a few days later decides to google you and then buy, Google Analytics is unable to know if the sale was generated by Facebook."

To answer this question in a scientific way we decided to run an Incrementality Test on at a geographical level

How Meta Incrementality Testing Works

What is a Geo-based Incrementality Test?

The geo-based incrementality test is an analysis method that helps you understand the true impact of your Marketing campaigns

You can run incremental tests on Facebook / Meta, Instagram, Twitter and other platforms, channels and networks.

How does an Incrementality Test work at a geographical level?

A geo-based Incrementality test is a fancy A / B test that works by comparing the results between two groups of regions:

• The Control group (no changes applied)

• Test group (group in which a change is made)

In the test group we will make a change on the expense for a defined period of time we can:

• Decrease spending

• Increase spending

The duration of an experiment typically ranges from 10 to 30 days.

At the end of the experiment, the variation of the conversions obtained with the control group compared to the test group is measured.

From this data, we can derive the incremental impact of the investment in the channel and the related Cost per Conversion

Running a Meta Incrementality Test: Our Implementation

Incrementality testing implementation

Problem: Facebook attributes too many transactions to itself.

Solution: Perform an incrementality experiment and measure real impact.

Goal: measure how many sales Meta ads generated at what cost per order.

To find out what regions to target, what budget, and how long to run our incrementality experiment, we used Cassandra.

Cassandra suggested to turn off the investments on Meta in Lazio and Piemonte for 10 consecutive days.

The investment should've to remain steady in the other regions.

But how did it end?

At the end of the 10-day test, we compared the variation in transactions in Lazio and Piemonte with the rest of the regions where we continued to spend on Meta ads.

Result:

We generated 120 fewer orders than we would have generated if we hadn't made any budget changes.

The actual Incremental Cost per Conversions was € 2.07.

4 times higher than that measured by Facebook Ads.

Conclusion

Meta Ads attributed to itself 4 times the number of transactions it actually generated.

Next steps:

Launch an incrementality test on the Google search and Google video channel to measure the CPO

FAQ

Does Meta Ads over-report conversions?

Yes. Meta's native attribution counts conversions using view-through and click-through windows that overlap heavily with organic purchases — customers who would have bought anyway. In this case study, Meta reported a CPO of €0.54 while a geo-based incrementality test measured the true incremental CPO at €2.07: a 4x discrepancy. The only way to quantify the real gap for your specific campaigns is to run a controlled incrementality experiment.

What is Meta incrementality testing?

Meta incrementality testing is a controlled experiment that measures how many conversions Meta Ads actually caused — not just correlated with. It works by pausing or reducing Meta spend in a set of test regions while keeping spend unchanged in control regions, then comparing conversion rates between the two groups. The difference represents the true incremental lift generated by Meta.

Why can't Google Analytics measure Meta's real impact?

Google Analytics only attributes conversions to the last click or the click within a defined attribution window. It cannot capture view-through conversions or the halo effect of users who see a Meta ad, don't click, and later convert through a different channel. This systematic blind spot causes GA to undercount Meta's attributed conversions — while Meta's own reporting overcounts them. Incrementality testing is the only method that bypasses both distortions.

How long does a geo-based Meta incrementality test take?

A standard geo incrementality test runs for 10 to 30 days. Shorter windows reduce statistical confidence; longer windows increase the risk of confounding events (seasonality, promotions). In this case study, 10 days in two Italian regions (Lazio and Piemonte) produced a statistically meaningful result.

Can incrementality testing be run on channels other than Meta?

Yes. The same geo-based methodology applies to Google Search, Google Video, TikTok, and any channel where spend can be selectively paused by geography. The next step in this case study was running the identical experiment on Google Search and Google Video to measure their true incremental CPO.