Incrementality Testing: How to Measure Causal Lift

Incrementality testing measures the conversions or revenue caused by a marketing activity. It compares a treatment group that receives the activity with a control group that does not. The difference between their outcomes is the incremental lift.

That makes incrementality useful when an ad platform reports plenty of conversions but cannot answer the harder question: how many of those people would have bought anyway?

The basic incrementality test

A sound test has four parts:

1. Define one business outcome, such as completed orders or qualified leads.
2. Split comparable people or regions into treatment and control groups.
3. Change one marketing condition for the treatment group.
4. Compare the conversion rate or revenue between the groups.

Suppose 10,000 people in the treatment group produce 600 purchases and 10,000 people in the control group produce 500. The campaign produced 100 incremental purchases.

The conversion lift is:

(treatment conversion rate - control conversion rate) / control conversion rate

In this example, the result is (6% - 5%) / 5% = 20% lift.

Incrementality percentage is sometimes reported as the share of treatment conversions that were incremental. That calculation uses the treatment rate as the denominator. State the formula beside the result so finance, marketing, and vendors do not compare two different definitions as if they were one metric.

Common test designs

User holdout tests

Randomly assign eligible users to treatment and control groups. This design can produce a clean comparison when a platform or owned channel can keep the groups separate. It works well for email, lifecycle messaging, and ad platforms with supported lift studies.

Geo experiments

Group cities, regions, or designated market areas with similar historical behavior. Keep spend running in the control markets and change it in the treatment markets. Geo tests work for channels where user-level assignment is unavailable, including some offline media and broad paid campaigns.

Conversion lift studies

Several ad platforms offer managed lift studies. They handle audience assignment and report the difference in conversion behavior. These studies are useful, but each platform sees its own delivery and matched outcomes. A first-party outcome dataset gives you a consistent business result across tests.

What to decide before launch

Pick the decision first. A useful test answers a question such as, "Should we keep funding branded search at the current level?" A broad test of whether all marketing works rarely changes a specific budget line.

Then document:

• The primary conversion event
• The expected baseline conversion rate
• The smallest lift worth acting on
• The planned sample size and test duration
• The treatment and control assignment
• The rule for stopping or extending the test
• Other campaigns or promotions that could distort the result

Run a power analysis before launch. A test with too little traffic can return a noisy estimate that looks decisive only because the dashboard rounded it.

Why conversion data quality matters

The experiment can randomize perfectly and still fail if the outcome data is incomplete. Browser restrictions, ad blockers, duplicate events, and broken cross-domain journeys can affect how many conversions reach the analysis.

Hardal uses first-party server-side collection to capture the conversion outcome before it is sent to ad platforms or used in measurement. That does not create causality by itself. It gives the test a cleaner outcome dataset.

The size of the missing-data problem is measurable. arabam.com recovered 80.1% more PageView data and 22.8% more purchase conversions after moving critical events to server-side collection. A lift test built on the earlier browser-only feed would have started with fewer observed outcomes.

Incrementality, attribution, and MMM

Marketing attribution assigns credit across observed interactions. Multi-touch attribution can show the path from first visit to purchase. Neither method proves that an interaction caused the purchase.

Marketing mix modeling estimates how spend and external factors relate to business results over time. It covers channels without user-level paths, but its output depends on the data, controls, and model assumptions.

Use the three methods for different decisions:

Start with the channel where a wrong decision costs the most. Define the action you will take for a positive, negative, or inconclusive result before the test begins.