Every traffic source has some % of bot traffic, whether intentional or unintentional.
Not only can you not monetize bot traffic, some bots are malicious, whether to scrape your content (e.g. spy tools) or to inflate the number of real visitors, and having these clicks co-mingled with your legitimate traffic stats could mislead your decision making and apparent campaign performance.
With Thrive‘s Cloaker/Bot Filter module, we have some tools at our disposal that give us the ability to filter out bot traffic.
Difference Compared to Regular Redirect Rules
You may wonder how the Cloaker/Bot Filter is different than regular redirect rules that you create for a campaign.
With the Cloaker, you set filters to filter out unwanted traffic so that this filtered traffic data does not contaminate your “real” campaign stats, whereas the traffic going to redirection rules will be logged as a part of actual campaign data.
The Cloaker filters unwanted traffic to a designated “safe page” and filters the clicks to a separate report outside of the main campaign stats.
The Cloaker/Bot Filter also has an advantage in that it allows you to easily set the filters system-wide or to a list of specific campaigns. You can easily switch them on and off when desired. They are more nimble and easily maneuverable.
Warning: if you set a filter system-wide, do not set a Thrive campaign URL as the safe page. This creates an infinite loop!
Use the Pre-defined Filters
Thrive‘s Cloaker/Bot Filter comes with 2 pre-defined filters called “Known Bots” and “Known Bot Hosts & ISPs”.
I would recommend that you set these filters system wide. You don’t need to create any filter criteria for them. They are basic bot detectors that should already be able to filter out quite a few bots.
Filter Out Traffic with JavaScript Disabled
Another filter option we’ve more recently added to Thrive is the ability to detect traffic that has JavaScript disabled, most likely indicating that it’s a bot.
According to a recent study, only ~1% of all real visitors have JavaScript disabled. Only 0.2% of all visitors intentionally disabled JavaScript. Therefore, vast majority of consumers (and likely monetize-able people) have JavaScript enabled.
On the other hand, the vast majority of bots have JavaScript disabled and completely ignore the JavaScript code on a page.
This makes detecting JavaScript as a gauge to be a very effective way of filtering out bots.
It is a slightly more aggressive way of detecting bots, but you could easily set the “safe page” for these filtered traffic to another offer or Thrive campaign URL so that the 1% of real visitors are still monetized.
Note: an additional meta page is required to detect JavaScript, so it adds some milliseconds onto your redirect.
Filter Out Non-Residential Connections
Through Thrive‘s “Connection Type” filter, you can choose to filter out corporate, VPN and proxy connections, which essentially cover all non-residential connections.
This is a rather aggressive way of filtering, so you will have to use it on a case-by-case basis.
This filter looks at the ISP of the incoming traffic and will identify whether a connection is residential or non-residential.
You could be filtering out, for example, traffic from corporate offices (e.g. Microsoft, Amazon, etc.) where an employee is using his work connection to browse the web personally.
Using Whitelist Filters to Aggressively Filter Out Bots
The most aggressive way to filter out all traffic that you potentially don’t want in your actual campaign is to create whitelist filters.
Instead of blacklisting certain traffic and sending them to the “safe page”, you can whitelist just the criteria that you’re specifically targeting and only let that traffic through.
For example, let’s say you are buying Android traffic from Germany. If you create a whitelist filter with criteria Android and Germany, then only traffic matching those criteria will be let through. Everything else will be filtered out and sent to the “safe page”.
With our list of possible rules, you can make your filters more optimal over time as you get a clearer idea of the kind of traffic that comes in.
Thrive has perfect transparency into the raw data of clicks that come in. You can analyze the data and create effective rules to more and more precisely filter out the unwanted traffic.
Leave a Reply