Once Upon a time…
A B2C eCommerce that invested 3000€/month in Meta Ads, had the doubt that Meta was attributing more transactions to itself than it actually generated.
In fact, Meta said that it was able to generate customers at an average cost of € 0.54.
This information was almost too good to be true.
As one of the main channels he invested in, he wanted to find out if Meta was telling the truth about his reporting.
How could we find out if Meta was reporting true conversions and measure the real impact of that spend?
How many conversions came from Facebook (Meta) campaigns?
Well, the answer was not Google Analytics.
Google Analytics attribution models only measure how many people bought immediately after clicking on an ad.
“If a person looks at your ad, does not click and a few days later decides to google you and then buy, Google Analytics is unable to know if the sale was generated by Facebook.”
To answer this question in a scientific way we decided to run an Incrementality Test on at a geographical level
What is a Geo-based Incrementality Test?
The geo-based incrementality test is an analysis method that helps you understand the true impact of your Marketing campaigns
You can run incremental tests on Facebook / Meta, Instagram, Twitter and other platforms, channels and networks.
How does an Incrementality Test work at a geographical level?
A geo-based Incrementality test is a fancy A / B test that works by comparing the results between two groups of regions:
- the Control group (no changes applied)
- Test group (group in which a change is made)
In the test group we will make a change on the expense for a defined period of time we can:
- Decrease spending
- Increase spending
The duration of an experiment typically ranges from 10 to 30 days.
At the end of the experiment, the variation of the conversions obtained with the control group compared to the test group is measured.
From this data, we can derive the incremental impact of the investment in the channel and the related Cost per Conversion
Incrementality testing implementation
Problem: Facebook attributes too many transactions to itself.
Solution: Perform an incrementality experiment and measure real impact.
Goal: measure how many sales Meta ads generated at what cost per order.
To find out what regions to target, what budget, and how long to run our incrementality experiment, we used Cassandra.
Cassandra suggested to turn off the investments on Meta in Lazio and Piemonte for 10 consecutive days.
The investment should’ve to remain steady in the other regions.
But how did it end?
At the end of the 10-day test, we compared the variation in transactions in Lazio and Piemonte with the rest of the regions where we continued to spend on Meta ads.
Result: We generated 120 fewer orders than we would have generated if we hadn’t made any budget changes.
The actual Incremental Cost per Conversions was € 2.07.
4 times higher than that measured by Facebook Ads.
Results
Meta Ads attributed to itself 4 times the number of transactions it actually generated.
Next steps: Launch an incrementality test on the Google search and Google video channel to measure the CPO
