The concept of testing marketing content before committing it to a campaign is nothing new; marketers have been running focus groups and surveys for years to determine the content that has the most potential. While these tests can be effective in gaining qualitative insight on the strengths and pitfalls of various content, optimization 2.0 (the digital platform) can do far more! Optimization through A/B testing provides marketers with quantitative data comparing two versions of content. A/B testing is the process of creating a different version of content and releasing it randomly to half of visitors and gathering data on which version produces a better result. For example, let’s say a company released two versions of their landing page, one that featured a video and one that was a simple navigation window. If the objective of the landing page is to function as a tool to deliver visitors to the rest of the sight, the company would select the version of the landing page that had the lower bounce rate. A/B testing can also be used to test far more minute differences like the color of a font on the checkout page. This gives digital A/B testing an advantage as an optimization technique over a traditional technique such as focus groups as A/B testing provides quantitative data and can test finer differences in real time. Adobe’s 2014 Digital Marketing Optimization Survey shows that companies that use testing to optimize their content and in their decision making process are far more successful, with companies that have a culture of optimization demonstrating a 100% higher conversion rate.
One of the great stories of success through A/B testing is the Obama 2012 campaign. Anybody who gave the campaign their email address received (and may still be receiving) countless emails. These emails were tested on the basis of subject line, donation requested, general formatting and many others. An example of what a subject line test might have looked like (provided by Businessweek) is posted below:
The campaign was based upon continual analysis of data like this that fueled continual change. Regardless of whether or not it made sense, if something produced results in the testing phase it would be put into practice until it was no longer the top performer in testing. An example of this was a trend that the marketing team noticed of uglier email formats, such as obnoxiously highlighting important text, was producing greater results. As soon as this trend had run its course it was abandoned without hesitation. While this blind faith in data worked for the Obama 2012 campaign, that does not make it right for every company. A paper written by Microsoft on web experimentation warns against many of the tactics employed by the Obama 2012 campaign such as following trends that they did not fully understand and making decisions based on short-term impact. The Obama 2012 campaign was able to get away with this because they are not a business with long term objectives but an organization with one very short term objective. While marketers can learn much in terms of how to prioritize data in decision making and the extent to which testing can be done, they must remain cognizant of the differences between a business and a political campaign.
Another example of A/B testing success, provided by Hubspot, is that of Upworthy, a site designed to provide and encourage the sharing of videos. Upworthy observed the that sites with recommended content had higher rates of visitor engagement. While Upworthy was seeking a way to better provide content to their users, there was a concern that a recommended content feature would counter the objective of social sharing. The easy answer was to test a version with the recommendation feature against the current version. Upworthy found that the recommendation increased social sharing by 28%, making the choice obvious.
This afternoon I will get the privilege of chatting with Amelia Showalter, the mind behind the Obama 2012 campaign’s digital analytics, via Google Hangouts. I will update this blog with any noteworthy info I gain from the conversation.