Just when you think we’ve hit the limit on A/B testing, another article comes out showing how important a good testing program is for success in today’s multi-channel world. In this instance, the focus is on Netflix, covered in this interesting note by Daniel Frankel in FierceCable, which reports on a presentation by Gabriella Mirabelli.
In a world dominated by choice, not only between competing brands but between channels themselves, attention becomes a scarce commodity. Getting it, and keeping it, is essentially the business model for many organizations across multiple verticals that we wouldn’t typically see as competing with each other.
A brief look at our own lives shows this clearly. When we pull out our phones to pass the time while waiting in line or for a late friend, a small battle unfolds among social media, game developers, news outlets, and more – and our attention is the prize. This situation reflects a broader trend – when I finally get to relax on the couch after a long day of writing about A/B testing, which screen do I pick and what do I decide to do with it? (I’m skipping the option of talking to real people – that’s so 2016.)
How A/B Testing Helps Win The Attention Wars
In an environment like the one described above, businesses that want to compete successfully can’t make too many mistakes when it comes to ensuring users find something that catches their eye and then proceed to engage with it. That’s doubly true in situations – and this is particularly true in the media industry – where content is shared and found via social media platforms.
Put simply, we want to ensure that everything we do is as compelling as possible and ‘pulls in’ the user or viewer. And we don’t do that by guessing. We do it by analysis of real user data, and that in turn means A/B testing. Netflix is just the latest in a long line of successful organizations to adopt and promote the ‘test everything’ philosophy. Ultimately, internal arguments about anything that could be described as a ‘creative decision’ should raise alarm bells. Whenever you find yourself in this situation, stop and ask whether a properly conducted test couldn’t answer the question for you.
What is particularly interesting about the Netflix example is the level of detail involved: “For example, Netflix will show a test audience two versions of a thumbnail for original series “Orange Is the New Black”: one full of elements, the other only featuring star Tyler Schilling and a few text elements. The latter tested much better” Frankel reports. Whilst it is always important to remember that A/B testing goes far beyond the cliched examples of red v green buttons and so on, it is still necessary to be aware that the devil is very much in the detail, and that cumulatively the little things really DO matter.
But don’t be limited. This recent example of Facebook testing in action shows another successful organization that is both a) never satisfied and b) aware that even quite fundamental decisions around interaction design and information architecture can and should be tested out there in the real world. A/B testers everywhere – we salute you!