Proving a Channel Added Anything
I’ve lost count of how many times I’ve sat in a meeting with a founder who is being sold a “proprietary attribution model” that costs more than their monthly payroll. These agencies love to use big words to hide a simple truth: they are often just taking credit for sales that would have happened anyway. If you are currently staring at a dashboard wondering how to test incrementality without hiring a data scientist and a team of analysts, you aren’t doing it wrong—you’re just being lied to. Most of these “rigorous statistical frameworks” are just expensive ways to make a tiny bit of luck look like a masterstroke of genius.
I am not here to give you a lecture on multi-touch attribution or complex modeling. My promise is much simpler: I’m going to show you how to run a test that you can actually set up and monitor in a single afternoon. We are going to strip away the jargon and focus on the only metric that matters—did that specific dollar spend actually change the outcome? No funnels, no fluff, just practical math you can use to stop wasting your budget.
Ditch Complex Marketing Attribution Models for Real Truth

Stop letting agencies sell you on complex marketing attribution models that require a PhD to decipher. I see it constantly: a small firm spends thousands on a software suite that claims to track every single touchpoint, only to realize they’re just paying to see a digital map of what was already going to happen. If your customer was going to buy from you because they saw your sign on the way to work, a fancy multi-touch model isn’t “proving” your digital ad worked; it’s just taking credit for the inevitable.
Instead of chasing ghosts in a dashboard, focus on the only thing that matters: did the spend actually change the outcome? Forget the theoretical math and look at conversion lift studies that actually make sense. I want to see the delta between people who saw your ad and people who didn’t. If you can’t prove that the ad caused the sale, stop pretending it did. Don’t get bogged down in high-level academic theory; just find a way to isolate a small group of people who don’t see your ads and compare their behavior to the rest. That is the only way to find the truth without wasting your entire week on a spreadsheet.
The Lazy Guide to Control Group Methodology

Look, you don’t need a PhD in causal inference in advertising to figure out if your spend is actually working. Most agencies will try to sell you on complex conversion lift studies that require a data science team and a six-month window. That is nonsense for a business with three employees and a tight budget. You just need a way to see what happens when you stop doing something.
Here is the afternoon version of control group methodology: pick your most consistent channel—say, your weekly newsletter or a specific Meta ad set—and simply turn it off for a handful of people or a specific geographic area for one week. This is your control group. Compare the sales from the people who saw the ads against the people who didn’t. If the “no-ad” group still buys at the same rate, you aren’t driving growth; you’re just paying for customers who were going to find you anyway.
It’s blunt, and it’s not pretty, but it’s the only way to get a real answer. Stop obsessing over every single click and start looking for that clear delta in revenue between your active and dark groups. If the difference is negligible, stop wasting your money.
5 Ways to Test Incrementality Before Your Coffee Gets Cold
- Stop obsessing over “last-click” data. If someone searches for your brand name and clicks a paid ad, you didn’t necessarily win them; they were already looking for you. Test this by turning off your brand keywords for three days and seeing if your organic traffic picks up the slack.
- Use the “Holdout” method on your email list. Next time you have a promotion, send it to 80% of your list and leave the other 20% in the dark. If the 20% group buys at roughly the same rate as the 80%, your email campaign isn’t driving sales—it’s just bothering people.
- Run a “Geo-Fence” experiment. Pick two similar towns or postal codes. Run your ads heavily in one and zero ads in the other for a week. Compare the sales lift between the two. It’s crude, but it’s faster than hiring a data scientist.
- Check your “Natural Sales Baseline.” Look at your sales from a Tuesday three weeks ago and compare it to a Tuesday during a big ad spend. If the numbers are within a 5% margin, your ads are just noise, not a driver.
- Test your social media impact with a “Dark Period.” Turn off all boosted posts for 48 hours. If your lead volume doesn’t dip, you aren’t paying for growth; you’re paying for vanity metrics that look good in a report but don’t touch the bottom line.
The Bottom Line for Busy People
Stop paying for software that tries to guess which click caused a sale; it’s just expensive guesswork that hides the truth.
Use a simple control group—stop showing ads to one small batch of people and see if they buy anyway.
If you can’t check the math in an afternoon, you’re wasting time on a metric that doesn’t actually pay the bills.
Stop Guessing and Start Testing

Look, we’ve covered the ground. You don’t need a data science degree or a six-figure software subscription to figure out if your marketing is actually working. You just need to stop trusting those shiny, automated dashboards that claim every single click leads to a sale. Forget the complex attribution models that take weeks to build; just use a simple control group or run a quick geographic test to see the actual delta in your revenue. If you can’t set up a basic test in a single afternoon, you’re wasting time that should be spent on driving actual growth.
At the end of the day, marketing shouldn’t feel like a math exam you didn’t study for. It’s about making informed decisions so you can stop throwing money into a black hole. You don’t need to be perfect, and you certainly don’t need a massive department to get the answers you need. Just pick one channel, run one test, and look at the numbers. Stop overthinking the theory and start valuing the evidence. Now, go get those results.
Frequently Asked Questions
I don't have a budget for a control group—can I just turn off my ads for three days and see what happens?
Yes, you can. It’s called a “switchback test,” and frankly, it’s better than most expensive software. Turn the ads off for three days—ideally during a normal business week—and track your baseline sales. Compare that to the three days before you cut them. If your revenue doesn’t take a measurable dip, those ads were just poaching sales you would have gotten anyway. It’s fast, free, and tells you the truth.
How do I know if the sales I'm seeing are actually from my marketing or if people were going to buy anyway?
Stop looking at your dashboard; it’s lying to you. Most of those “conversions” are just people who were already planning to buy. To find the truth, pick one channel—say, your Meta ads—and turn them off completely in one specific geographic area (like a single city) for three days. If your total sales in that city don’t drop, your ads aren’t driving growth; they’re just taking credit for existing demand.
If I run a test and the numbers don't move, does that mean my entire marketing strategy is broken?
No. It just means that specific lever is broken. Don’t panic and burn the whole strategy down. If you spend £500 on ads and sales don’t budge, it doesn’t mean your brand is dead; it means those specific ads or that specific channel aren’t working. Treat it like a single bad round in a pub quiz—you lost the points, but you haven’t lost the game. Stop, pivot the spend, and try a different tactic tomorrow.
Is there a way to track this without spending four hours setting up a spreadsheet?
Yes. Stop trying to build a custom dashboard. If you aren’t using a tool that does the heavy lifting, just use the “Holdout” method. Pick one specific channel—say, your Meta ads—and turn them off entirely in one specific geographic region or for one specific segment of your email list for three days. Compare the sales in that group against your usual baseline. It’s messy, it’s blunt, and it takes twenty minutes. It works.


























