Reduce Campaign ROI Volatility: User Similarity Control

Jan 16 By Aziz Kamara

Just about everybody’s experienced this once they start building profitable campaigns: one day the campaign will be 100%+ ROI and the  next in the red.  While day-to-day volatility cannot be avoided due to seasonal changes, demographic changes, trend changes, inadequate sample sizing, and a whole host of issues, you can certainly reduce it.

What happened the day the campaign achieved 100%+ ROI?  The POF users that happened to see your ads clicked on your ads more and fill out the offer forms more.  Why did this not happen the day after?  If we control all other external factors, this can be attributed to the users from day 2 being uniquely different than the users from day 1, which may be due to your targeting being too broad.  Your target audience could be too different, causing them to react very differently to your ads, and it’s luck of the draw which targets you get on a given day.

Let’s illustrate this with an example.

Let’s hypothetically split test 2 different targeting parameters:

Group 1. All the good guys

Group 2. Darth Vader and his identical twins

Let’s assume we are marketing an offer for a free iPad.  Well, the good guys are going to behave differently: Skywalker loves iPads but Han Solo hates iPads (he prefers guns).  Depending whether or not you get Skywalker or Han Solo that day, your campaign could do really well or not at all.  While with Group 2, Darth Vader loves iPads (like son like father), and his identical twins aren’t really that different (they like operating the Death Star remotely with it).  They will all click on your ad.  Your campaign will be a lot more consistent.

Narrow Your Targeting

Conceptually, the more similar your user base in behavior, the less volatile their actions will be.  In our example, Group 2 is the extreme where everyone is basically the same person.  In real life, of course you will not be targeting one person at a time.  It’s an extreme to demonstrate that likeness equates to higher efficiency.  But if you can group your traffic into compartments of likeness, you can more efficiently discover and feed them what they want the most.  That’s why I always advocate to target your campaigns as narrowly as possible to the point you can still efficiently handle the amount of campaigns created.

Targeting narrowly does not only include demographics.  It also includes things like login count.  The same person behaves differently after they’ve logged into their account 50 times versus the first time. Anything that affects behavior should be considered.  And if you’re having trouble narrowing your female Caucasian demographic down, make the age ranges narrower.  People of the same age behave more similarly.  Targeting narrowly is also conducive to longer, more sustainable campaigns.

PS, if you’re wondering, all the good guys are dead due to overpopulation of Darth Vaders, so my case study campaign is killing it right now.

Edit: I need to add on to this post that if the targeting is too niche, there will also be increased volatility from day to day due to the lack of traffic. Said another way, day to day volatility increases as the data sample size gets smaller. This isn’t necessarily a big negative though, as we should compare our ads across the same amount of data not day to day.

Never miss a feature, product launch, or exclusive offer


  • Tom,

    Very interesting article!

    When you’re thinking about a niche, or how niche to go with your targeting, what types of signs or signals do you look for to find a balance between a big enough target group for adequate volume and niche enough to have longevity?

    Thank you for another fantastic post!

  • That’s tough to say because it depends on how well/how much time you can manage a lot of campaigns. Ideally, you would be as niche as possible, but that would create too many campaigns normally for anyone to manage. It really comes down to how much your time is worth.

    In the beginning, when you are learning and conserving resources, you might want to create a bunch of niche campaigns so they have a higher probability of success. As you become more successful, you can decide if doing that is worth your time or if you should go after broader campaigns.

  • Leave a Reply

    Your email address will not be published.