What’s so bad about A/B and multivariate testing?
Ever since John Wanamaker quipped, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half,” marketers and advertisers have been trying to predict which ads will work with what audience. For a long time, marketers have had to use less-than-optimal strategies, like A/B and multivariate testing, to (partially) accomplish this goal.
A/B testing is simple: If you’re testing puppy ads and kitten ads, and 70 out of 100 people prefer puppies, everyone going forward sees that ad. That’s great, until you realize that you’re isolating the 30 percent of your audience who prefer kittens.
Multivariate testing involves testing different variations of that creative to see what combinations perform the best. For example, you could test six variations of a creative, all with different calls to action. However, testing all of those variations can require an immense amount of inventory and budget to get statistically significant results.
How is Machine Learning Different?
Thankfully, machine learning technology has evolved to make it possible to predict what creative will resonate with each individual seeing it, all while the campaign is running and without user input.
Here’s an example of how it works, using our puppy and kitty ads example
Let’s say that the machine knows that women ages 25-34 in cities, and men ages 35-56 in the suburbs have clicked on pet food ads in the past. Machine learning technology will show people matching these attributes the brand’s puppy and kitty ads. If, during this campaign, it learns that women in cities are more likely to click on the puppy ads, while men in suburbs are more likely to click on the kitty ads, it will start serving more of those ads to more people matching those sets of attributes.
Then the brand learns that it’s not just women in cities, but women everywhere love puppies. Furthermore, they’re more likely to click if they’re on a tablet vs. a smartphone. Therefore, the machine lessens the importance of location data and starts serving more ads to more women on tablets. Likewise, it learns that while the majority of men in the suburbs on iOS devices prefer the kitty ads, those on Androids actually click more on the puppies. Therefore, it starts serving men with Androids puppy ads, and men with iOS devices kitty ads.
The technology solution continuously tests and applies these learnings based on huge amounts of data that are analyzed in microseconds, refining the results until each individual impression, not just the majority, is served the creative most likely to produce the desired result.
Does it Actually Work?
An American multinational retail corporation that operates a chain of hypermarkets, discount department stores and grocery stores worked with LiveIntent to optimize its in-email marketing offers. In addition to having the world’s largest selection of online shopping essentials, the retailer is consistently looking to increase site traffic and email revenue.
The client tested marketing offers leveraging LiveIntent’s Optimization Engine. LiveIntent began the month by trafficking the marketing offers with no optimization, then switched all offers to utilize the LiveIntent algorithm to set a max clickthrough rate (CTR) strategy.
Here’s how it works
1. When a subscriber opens an email from the retailer, the LiveIntent platform facilitates a real-time auction for the available impressions between the retailer’s marketing offers, its digital agency’s advertisements and LiveIntent advertisers.
2. All offers and ads are escalated based on probability of engagement, per ad unit, per email opener. Variables such as campaign performance, impression/frequency goals, device, exposure, gender, behavior, bid price and browser are considered.
3. LiveIntent’s Optimization Engine looks at each person who opens an email to use the CTR optimization.
4. Finally, an impression for the predicted offer or ad is purchased, based on the likelihood of the email opener engaging with the ad.
This campaign resulted in 138 percent more clicks on offers, 105 percent greater CTR and 88 percent increased revenue per thousand opens.
To learn more about working with LiveIntent, visit our Contact Us page to learn more.