When venturing to answer the burning questions about attribution and the best place to spend the next advertising dollar, advertisers often turn towards incrementality testing in the form of exposed and control groups. This is especially true of digital advertising channels due to the ease of ghost bidding methodologies for executing exposed vs. control studies.
Five plus years ago, this experimental approach to attribution modeling was considered revolutionary and represented the scientific gold standard. In today’s marketing-sphere advertisers should reconsider the reliability of this approach and the validity of its results due to key elements that have emerged over time: identity, cross-platform comparisons, and scale.
Maintaining the sanctity of any exposed and control experiment requires that the two groups be mutually exclusive: consumers exposed to an ad never appear in the control group. Should a consumer in the control group be exposed to an advertisement, the base conversion level for the control group may increase on the effectiveness of the ad, thereby diminishing the incremental lift of the exposed group over the control and invalidating the experiment. Keeping the exposed and control groups independent of each other is most often the job of third-party tracking but may also be determined by logged-in users as in the case of Facebook, for instance.
Perhaps the greatest threat to the validity of third-party tracking for determining discrete exposed and control groups is iOS’ move towards a user-centric data privacy approach. Mozilla’s Firefox has already begun blocking third-party cookies and Google is phasing out third-party tracking cookies in Chrome browsers. Additionally, consumer privacy measures like CCPA/GDPR provide further headwinds for first and third-party tracking.
Third party tracking is not going away but until a different method for tracking people across the web is consistently adopted, it is unlikely the discrete exposed and control groups can be reliably maintained. Alternatively, adopting a geographic based approach to incrementality measurement affords the same scientific rigor but with more reliable execution.
In cases like Facebook where first party tracking may still provide a reliable vehicle for discrete exposed and control groups, one should consider the usefulness of such a result in the context of incrementality results obtained for other channels by other means. Said differently, preserving the integrity of incrementality results is best accomplished by using the same methodology, perhaps a geographic based approach, to collect results across all platforms. Doing so eliminates the risk of introducing bias into the experiment from methodological differences and facilitates more reliable cross-platform comparisons.
The final factor to consider when embarking on incrementality testing is the overall marketing mix and the test channels’ share of the total. Today’s marketing mixes are leveraging an increasing number of different acquisition strategies making parsing out the contribution of a single channel exceedingly challenging. To expect any individual channel to drive incrementality that is significant enough to rise above the noise of the overall marketing machine might be unrealistic. This is not to say that the channel in question does not contribute to incremental acquisitions, but rather that its impact is not measurable; these are two different things. By executing geographic based incrementality tests, advertisers can tweak the media mix less expensively with less risk to the overall business while affording each test channel an opportunity to be measured.
In summary, despite a growing buffet of advertising measurement solutions to chose from, the question of attribution remains at the forefront of unanswered questions. Incrementality testing is a great way to further our understanding of the overall attribution picture so long as the test is executed with care and precision. Otherwise, incrementality testing can be counterproductive, contributing instead to the confusion spurred by a collection of opposing signals to sort through. Geographic based lift testing allows for clean execution of incrementality testing for attribution, allows for measurement of both online and offline channels, and standardizes to a single measurement approach enabling fair comparison of results across platforms.