Just when you think we’ve reached ‘peak A/B testing’, along comes another article reminding us just how central a proper testing program is to success in today’s multi-channel world. In this case the business is Netflix, subject of this interesting note from Daniel Frankel in FierceCable - reporting in turn on a presentation by Gabriella Mirabelli.
In a world dominated by choice, not only between competing brands but between channels themselves, attention becomes a scarce commodity. Getting it, and keeping it, is essentially the business model for many organizations across multiple verticals that we wouldn’t typically see as competing with each other.
A moments reflection on our own lives confirms this. When we take our phone out of our pocket in order to kill five minutes whilst we wait in line or for a tardy loved one to arrive, a small war is waged between social media platforms, games companies, traditional media organizations and many, many more - and our eyeballs are the prize. And what is true in that specific scenario is true elsewhere - when we sit down on the sofa after a long day’s work writing about A/B testing, to which screen do I turn and what do I choose to do on it? (I am leaving the possibility of talking to actual people out of the analysis - that’s so 2016)
How A/B Testing Helps Win The Attention Wars
In an environment like the one described above, businesses that want to compete successfully can’t make too many mistakes when it comes to ensuring users find something that catches their eye and then proceed to engage with it. That’s doubly true in situations - and this is particularly true in the media industry - where content is shared and found via social media platforms.
Put simply, we want to ensure that everything we do is as compelling as possible and ‘pulls in’ the user or viewer. And we don’t do that by guessing. We do it by analysis of real user data, and that in turn means A/B testing. Netflix are just the latest in a long line of successful organizations to adopt and promote the ‘test everything’ philosophy. Ultimately, internal arguments about anything that could be described as a ‘creative decision’ should raise alarm bells. Whenever you find yourself in this situation, stop and ask whether a properly conducted test couldn’t answer the question for you.
What is particularly interesting about the Netflix example is the level of detail involved: “For example, Netflix will show a test audience two versions of a thumbnail for original series “Orange Is the New Black”: one full of elements, the other only featuring star Tyler Schilling and a few text elements. The latter tested much better” Frankel reports. Whilst it is always important to remember that A/B testing goes far beyond the cliched examples of red v green buttons and so on, it is still necessary to be aware that the devil is very much in the detail, and that cumulatively the little things really DO matter.
But don’t be limited. This recent example of Facebook testing in action shows another successful organization that is both a) never satisfied and b) aware that even quite fundamental decisions around interaction design and information architecture can and should be tested out there in the real world. A/B testers everywhere - we salute you!
