I'm helping brands eliminate their wasted marketing spend 🔥. Follow me for essays on marketing effectiveness in an omni-channel world.
We’ve seen this a few times: brands come to us and say something like, “Look, we ran an incrementality study and there was a 3% lift and the result was statistically significant!”. Sadly, the results they shared are meaningless without more information. Because you don’t care if a marketing channel has “any positive impact”, you care if it has a profitable impact. The problem with a number like “lift %” is that it doesn’t take into account how much money you spent to get that lift. The question you want an actual answer to is, what was the incremental ROI or the incremental CPA number? And you’re still not done! You also need the uncertainty intervals associated with the results of this experiment. We’ve had customers tell us that an experiment said a channel had “an incremental CPA of $75”, but it turns out that the uncertainty intervals range from a CPA of $25 to $750. When you’re running lift tests – and especially if you’re running them alongside an MMM – you really don’t want to be led astray and bias your budget allocation decisions. Thomas Vladeck and I are going to dig into all things lift tests and MMM during our next Marketing Science Office Hours. The session is on October 30th and the link to sign up is in the comments. I’d love to see you there!
VP Business Analytics Manager
2wWould love to hear about it!