a vs b

Recently, I’ve found myself writing a lot around the subject of A/B testing and how it can be invaluable in helping to create a site or app which converts. Also known as split testing, it basically involves taking an element on a design and changing it slightly, so that two different pages are created with a slightly different element on each.

These elements can be any one thing that prompts a user to take action, such as a call-to-action button, or it can involve a colour change or the use of different images and layouts. An A/B test, in order to be successful, must have a large enough ‘sample size’ and must run for long enough to gather accurate results. If the sample size is too small, or if the tester runs it until they see a positive results then stops, it’s likely that the test will throw up misleading results.

Testing more than one element at a time can also lead to confusing and inaccurate results; it’s always best to remember that less is more when it comes to split testing.

A/B testing can and does provide useful information though that can really help to convert visitors into paying customers.

What Elements Can You Test

When you’re looking at a web page, more or less every element that you see can be tested.

These include:

  • Buttons
  • Headlines and copy
  • Forms
  • Images
  • Site colours
  • Navigation
  • Layout
  • Landing pages

With the latter, a landing page is often set up as a CTA, so it’s highly useful to run a test to see which returns the best results.

A couple of years ago, Performable ran a test to see which button colour returned the best results, using a red button and a green. The results were surprising, not least because most people associate red with delete and green with go.

Red performed the best and returned a 21% increase in conversions. The rest of the home page was left unchanged and just the button colour changed – you can see the different pages in the image below.

A/B test by performable

The somewhat surprising results of this test proved one thing: it’s all but impossible to second guess how your users will behave when they land on your site. This is why split testing is invaluable. Even simple things like the way that you word things on your buttons and navigation can make all the difference. Just changing words so that they are a little ‘friendlier’ can make be the difference between a visitor clicking through or moving on.

Whilst to many of us, navigation is something that we don’t pay a whole lot of attention to, it’s possibly one of the most important parts of a site.


  • Tells us what the site contains at a glance
  • Shows us how to use a site
  • Guides the visitor on where they need to go
  • Gives a sense of reassurance to the visitor as it’s a constant (or should be)

In another test carried out last year, changing the wording on one navigation element returned a 47.7% increase in clicks on that element.

a/b test formstack

As you can see, the wording was changed from ‘why use us’ to ‘how it works’ and the site also found that it experienced a 21% rise in conversions after carrying out this and other tests following a site redesign.

“I think one of the early discussions (or disagreements) was what we were going to display in the main navigation. The main nav is so important in establishing the flow of traffic and the focus of the site. We all agreed on highlighting our forms, features, examples, and pricing, but we were not sure what would be the best page to use as a lead,” saidJeff Blettner, a web designer and conversion optimisation specialist at Formstack.

Don’t Give Too Much Choice

A more recent example was carried out by software firm Power Admin, which decided to give A/B testing a go following reading about the ‘jam test’, in which people who were presented with lots of choices didn’t buy as much as those that were presented with fewer.

The test basically found that when presented with too many choices, people actually made none and simply left. Power Admin took a look at its own site and decided to apply the principle to its navigation system, which was presented via lots of textual links, as shown in the image below.

Textual navigation links lots

So as you can see, there is a lot of choice there and this, potentially, could mean that as the user is presented with too much, they decide not to bother and leave the site. With this in mind, the navigation was changed to a one line system, as shown below.

screenshot nav system

The test ran from the beginning of July and is coming to an end as we speak. According to William Thompson, Marketing Manager, the test was a resounding success and whilst the new page didn’t gain as many visits as the old, the site gained 12.3% more conversions.

“The 12% would amount to about 70-80 extra conversions in a month, which could conceivably make a fairly sizeable improvement in our sales,” William told me in an interview.

Consider Your Users

UX, or user experience, is a field that’s been around for a long time but has seen a renewed interest in recent years. This is because it’s becoming increasingly important to please the modern internet user who now expects certain things from a site. Navigation is of course an extremely important aspect of every site and getting it wrong can have a serious impact on your bottom line.

With this in mind, and considering the above examples, you should take a look at your navigation system to determine if it’s effective and recognisable to the user. By this I mean that it should at a glance give a strong indication as to the content of each page. Judging by the above, you should also consider limiting choice, giving consideration to colour and choosing wording carefully if you’re to get the most from it.

Read more: