Your Best-Performing Ad Might Be Lying to You

Dropbox turned off entire ad channels for a month, just to see what would actually happen.

Their search ads looked like they were producing a 2x return. Solid, by most standards. But when Dropbox measured the real, causal return, meaning what actually happened because of the ad rather than what happened around it, the number came out to 0.92.

Underwater. Their best-looking channel was losing them money.

Attributed return and causal return are not the same thing

Most business owners have never separated these two numbers, because most ad platforms only show you one of them.

Attributed return counts the last thing someone clicked before they bought. If a customer searches your business name and clicks your ad, the platform credits that ad with the sale, even if that customer was already planning to buy from you and would have found you anyway.

Causal return asks a harder question: what would have happened if that ad never existed? Would the sale have happened anyway?

Dropbox's search ads looked great by the first measure. They lost money by the second, because a large share of those "conversions" were people who were already searching for Dropbox by name and were going to click through and buy regardless of whether an ad was sitting there or not. The ad wasn't creating the sale. It was just standing in front of a sale that was already happening.

Dropbox pulled $25 million out of that kind of low-performing spend. Their business got healthier, not weaker.

You don't need their budget to learn their lesson

You're probably not running a month-long controlled experiment across every ad channel you use, and you don't need to.

Pick your smallest, cheapest channel. Pause it for two weeks. Watch what actually happens to your calls, bookings, or sales.

If nothing changes, you just found money you didn't know you were wasting. If something does change, you just confirmed that channel is doing real work, which is worth knowing too. Either way you learn something your monthly report was never built to tell you.

How this shows up for a business your size

Kyle runs Sentinel Home Inspections in the Dallas-Fort Worth area. A real estate agent referred a client to him for a condo inspection, which happens all the time in his business. Most people would take the referral and move on.

Kyle asked her a simple question: how did you find me?

She said she'd asked AI for an inspector recommendation, and his business came up first.

No ad. No click. Nothing in his analytics dashboard would have ever shown him that. The referral looked like a normal referral. The real story was that his business is showing up as a direct AI recommendation, something worth knowing and worth protecting, and the only reason he found out was that he asked.

That's the Dropbox lesson at a scale any business can run. The dashboard tells you what got clicked. It doesn't tell you what actually worked.

Five things to do about it

1. Run a smaller version of the same test. Pick your cheapest, smallest ad channel and pause it for two weeks. Watch whether your real numbers move. That's your causal return test, no data science team required.

2. Separate brand-name searches from category searches in your ad spend. If you're paying for an ad to catch someone who searched your business name directly, check whether that sale would have happened without the ad at all.

3. Ask where the customer actually started, not just where they converted. You can't run Dropbox's exact experiment, but you can ask one question every time someone books or gets referred to you. Kyle's question took ten seconds and told him something his reports never could.

4. Watch for the channel that always looks good on paper. Search ads look great in almost every report because they catch people at the exact moment they're ready to buy. That's also what makes them the easiest channel to overstate. If one channel's numbers look too clean, test it first.

5. Redirect the budget, don't just cut it. Dropbox didn't just save $25 million, they moved it. If a test shows a channel isn't creating real demand, put that money toward the things that build demand before someone searches at all, your Google Business Profile, your reviews, the content that answers questions before anyone clicks anything.

Ask the extra question

Your dashboard shows you what people clicked. It doesn't show you what actually worked, and it definitely doesn't show you when your business shows up as a direct AI recommendation with no click involved at all.

The businesses that ask the extra question find money and visibility their reports were never built to show them.

Have you ever tested what happens when you turn a channel off, or have you just been trusting the report?

Want to know what's actually working in your marketing? Book a call and we'll walk through it together.