Tensor LabsTENSORLABS

Half your ad spend buys customers you already had

Geo-holdouts measure what spend causes, not what it stands near

July 20, 20264 min read4 sectionsBy Ahmed Abdullah
Half your ad spend buys customers you already had

Introduction

On the left screen, the platform dashboard: return on ad spend, 4.2, the number that had defended this channel's budget in every planning cycle for two years. On the right screen, the readout from a four-week geo experiment the growth lead had quietly run: incremental return, 1.1. Same channel, same weeks, same customers. The room looked from one screen to the other and did the subtraction nobody wanted to say out loud: roughly three quarters of what the dashboard called "return" was revenue that would have arrived anyway.

The channel was not acquiring those customers. It was standing next to them at checkout. Attribution tells you who was near the sale. Incrementality tells you who caused it.

Two different questions wearing the same number

This is not the tracking-pipeline problem, and fixing your collection does not fix it. Perfectly deduplicated, server-side, consent-clean attribution still answers only one question: which touchpoints appeared along the path to conversion. Budget decisions need a different question entirely: what would have happened to revenue if this spend had not existed. That is a counterfactual, and no amount of pixel hygiene produces a counterfactual.

The channels that exploit the gap are always the same ones. Branded search intercepts people typing your own name, the single strongest already-coming signal a human can emit. Retargeting bids on people who just visited. (Retargeting exists to take credit for people who were already coming back. It is extremely good at it.) Both post spectacular attributed ROAS, both sit at the top of the budget, and both are exactly where incrementality tests find the least lift, because proximity to conversion is what they optimize and proximity is what attribution pays.

The experiment that answers the real question

The method is the geo-holdout, and its virtue is that it needs no platform's cooperation and no tracking sophistication at all. Choose a set of markets, turn the channel off, or sharply down, in some of them for two to four weeks, and compare revenue against control markets. If the spend causes revenue, the treatment geos fall. If they do not fall, the dashboard has been narrating coincidence.

The craft that separates a clean read from an expensive shrug lives in three places. Market matching: control geos are built from pre-period trends, a synthetic control assembled to track the treatment markets' history, so the comparison inherits seasonality instead of being wrecked by it. Power analysis before launch: a test too small or too short to detect a plausible effect produces a confident-looking zero, so the design starts from how many conversions are needed, not from how many weeks feel comfortable. And a decision rule agreed in advance, because a readout that arrives without a commitment about what it changes becomes a slide, and slides do not reallocate budgets.

Run it as a calendar, not a stunt: one channel per quarter under test, results feeding budget priors, so over a year the whole mix gets audited by experiment instead of by attribution's self-report. We put this machinery in place for a subscription brand, matching, power math, automated readouts, and the first cycle found their two largest channels sitting at incremental returns near one while a mid-tier channel everyone had been starving tested at three and a half. The money moved. Revenue did not miss it, which was the entire point.

A channel that cannot survive a holdout was never a growth channel. It was a toll booth on traffic you already owned.

The honest caveats: geo tests need enough volume to read, small brands may need longer windows or coarser designs, and switching costs are real, some channels rebuild momentum slowly after a hard off. Design around all three; none of them rescues a channel whose lift was fiction.

One test before the next budget cycle

Pick the channel with the highest attributed ROAS and the strongest smell of already-coming traffic, branded search is the usual suspect, and put it through one properly powered four-week holdout before you re-approve its budget. The result will either defend the spend with evidence attribution can never provide, or free up the single largest recoverable line in your marketing P&L.

TensorLabs builds incrementality infrastructure, geo matching, power analysis, always-on measurement calendars, so budget follows cause instead of proximity. If a big attributed number in your mix has never been tested, write to us and name the channel. That sentence alone is usually enough to say what the test would cost and what it might return.