According to researchers from Southern Methodist University and University of Michigan, there are a lot of limitations in A-B testing of online ads and it can have significant effects on ad performance. In A-B testing, the company creates two ads of categories A and B. The ads are then personalised to audiences according to their preference. The two categories of ads are made to target specific consumers by placing the specific ads in front of audiences with similar interests. But the study by researchers find that A-B testing is not delivering what it should to marketers and this creates unreliable conclusions in ad performance.
The study introduced the term divergent delivery in which platforms like Google and Meta target specific users with different types of ads. The problem arises when the ads get mixed during A-B testing and appear on the wrong algorithm on a platform. The researcher of the study, Braun, says that when an ad is performing well only because it is appearing more due to the algorithm as compared to the other ad, it creates problems. Similarly, if an ad is not doing well, it can all depend on how many users are seeing it instead of what's actually in the ad content.
When the companies are large, ad targeting can cause a lot of impact and right ads to the right audience can provide them with a lot of value. Ad placement doesn't only depend on the monetary value of an ad bid. User-ad relevance and ad content also have equal importance. Ad-relevance affects auction results as well so when ads are being placed on a specific platform, the platform decides ads for specific audiences. This means that advertisers do not know that much. It is still not confirmed how algorithms determine user-ad relevance, but the study warns that marketers who rely on A-B testing to make their marketing strategies do not do that.
In conclusion, A-B testing is not reliable for analysis of ads. The researchers say that there isn't any technical problem in A-B tests, that's just how they operate because they tell marketers how to maximize their performance. Marketers should be aware of the limitations in A-B testing so they can make their ad campaigns that can actually help reach online ads to the right audience.
Image: DIW-Aigen
Read next: PayPal Phishing Scam Tricks Users with Authentic Emails: Are You at Risk?
The study introduced the term divergent delivery in which platforms like Google and Meta target specific users with different types of ads. The problem arises when the ads get mixed during A-B testing and appear on the wrong algorithm on a platform. The researcher of the study, Braun, says that when an ad is performing well only because it is appearing more due to the algorithm as compared to the other ad, it creates problems. Similarly, if an ad is not doing well, it can all depend on how many users are seeing it instead of what's actually in the ad content.
When the companies are large, ad targeting can cause a lot of impact and right ads to the right audience can provide them with a lot of value. Ad placement doesn't only depend on the monetary value of an ad bid. User-ad relevance and ad content also have equal importance. Ad-relevance affects auction results as well so when ads are being placed on a specific platform, the platform decides ads for specific audiences. This means that advertisers do not know that much. It is still not confirmed how algorithms determine user-ad relevance, but the study warns that marketers who rely on A-B testing to make their marketing strategies do not do that.
In conclusion, A-B testing is not reliable for analysis of ads. The researchers say that there isn't any technical problem in A-B tests, that's just how they operate because they tell marketers how to maximize their performance. Marketers should be aware of the limitations in A-B testing so they can make their ad campaigns that can actually help reach online ads to the right audience.
Read next: PayPal Phishing Scam Tricks Users with Authentic Emails: Are You at Risk?
