Optimising conversion with A/B testing: A guide for eCommerce businesses

Optimising conversion with A/B testing: A guide for eCommerce businesses | MollieOptimising conversion with A/B testing: A guide for eCommerce businesses | Mollie
Nick Knuppe
Product Marketing Manager
Customer-centric marketer and Go-To-Market perfectionist.

A/B tests can help boost your eCommerce business’s conversion rate. But how does A/B testing work and where should you start testing? This guide shows you the ins and outs of A/B testing, including a look at some common mistakes and a rundown on which A/B tests are most beneficial for your online shop.

What is A/B testing?

A/B testing is a tool for finding out how your website, online shop or other online channels can best appeal to your potential customers. During an A/B test, you test two different versions of a single page at the same time to see how differences in design features influence your customers’ behaviour. These might include differences in:

  • colours

  • placement of call-to-action (CTA) buttons on screen

  • text size

  • tone of voice

  • page navigation

You present both versions (A and B) to different groups of users. After that, you analyse how users behave on each version of the site. Based on a number of criteria, such as view time or bounce rate, you can determine which version is better suited for achieving your business goals.

Why is A/B testing important in eCommerce?

A/B tests (also known as split tests) are a common tool used in online marketing. They offer companies a way to test different versions of their newsletter, social media ads or Google ads. A/B testing is equally important in eCommerce. Using the right tests, you can quickly improve your online shop’s conversion rate and increase sales.

A/B testing: An example from eCommerce

Companies achieve a high conversion rate by turning more potential customers into actual paying customers. This often takes more than simply having good products and offering extensive information. Your site’s design and CTA buttons are just as important to encouraging visitors to make a purchase. The size, colour and position of the CTA buttons all can influence whether potential customers click on them. A/B tests are an ideal way of determining what works best. They help you to answer many different questions about the layout of your online shop, such as these:

  • Are the designs of the headings and images attractive enough?

  • Is the text in the breadcrumbs easy to read (an adequate font size, length, etc.)?

  • Do your customers prefer to place orders using one-page checkout or multi-page checkout? 

  • Are all the costs immediately visible, or do potential customers abandon the checkout process because shipping costs are hidden and only appear later in the process?

These are all perfect scenarios for using A/B testing, which lets you continually test and optimise changes you make to your online shop. The version that results in stronger sales is the right choice for your shop.

A/B testing on your online shop: How does it work?

No matter which industry you’re operating in, improving your conversion rate is always your main priority as an eCommerce retailer. Here’s a step-by-step guide to performing a successful A/B test to help boost conversion on your site.

Step 1: Collect data

Before starting your A/B testing, conduct some analyses to pinpoint the problems that are leading to a low conversion rate. Many analytics tools, such as Google Analytics, show you, for example, when a shopper places items in the shopping cart but then abandons the purchase at a certain point. It’s essential to put yourself in your target group’s shoes and figure out what might be deterring them from completing the order. Heat maps and usability tests are some other useful tools to apply at this point.

Step 2: Set priorities and goals

A/B testing on your online shop: How does it work? - Step 2: Set priorities and goals | Mollie

Once you’ve identified where the obstacles lie, it’s time to set priorities and clear goals for your A/B testing. It helps to start by formulating some hypotheses. Ask yourself questions like these:

  • Which element of my site has the greatest influence on whether the shopper decides to make a purchase?

  • If I change that element, what do I expect will happen?

Step 3: Conduct the test and analyse the results

To ensure that the results are clear and valid, it’s common for the A/B test to last for a long time. In other words, you want to make sure that your A/B test identifies factors that really are significant. The lower the traffic to your site is, the longer you’ll want the testing period to be. If your testing reveals a clear difference in performance between the two versions of the site, then this serves as a basis for further testing in which you change other elements. Remember that A/B testing is not just about changing one element; it’s about changing multiple elements and verifying how this affects performance. Otherwise, you have no way of knowing exactly which element is the most decisive for increasing your conversion rate.

It’s absolutely possible to test multiple alternative versions of an element at the same time. That means you’re not just testing A and B, but also C, D and so forth. Once you determine that one version is clearly performing worse than the others, it’s a good idea to disable that version and direct traffic to the others for the remainder of the test period. Use an A/B testing runtime calculator to check in advance how many versions you can run simultaneously.

A/B testing tips for online shops: Where do I begin?

There’s no standard rule that determines which element of your online shop should be tested first. Each eCommerce retailer should perform some preliminary analyses to figure out which elements of their sites have the greatest potential for optimisation. In general, it’s a good idea to use split tests on the following elements, which are all known to have a strong impact on conversion:

  • Headings: Whether short and to the point, or long and detailed

  • Images: Size, position, style, number, background images or galleries

  • Selling points: A neutral list of product functions or a list of key advantages for the customer

  • Price: Visible discounts, sale dates and shipping costs

  • Payment: Selection of payment methods, length of the payment process

  • Forms: Position, wording, design, optional fields, sequence of fields

If your analyses lead you to examine multiple relevant changes at once, the following tips can help you perform your first A/B tests.

Recurring elements on your site

On any website, there will be some elements that recur on nearly every page, such as these:

  • Header

  • Footer

  • Site navigation

  • Search bar

  • Chat

  • Sale banner

The table below shows some possible A/B testing tools to use on your online shop:

Page element Optimisation potential Version A Version B Version C (if needed)
Header In some cases, a ‘sticky’ header can improve the performance of your website. So, menu items at the top of the page scroll along with the user, so they are always easily accessible. Fixed-position header Sticky header Flexible header
Tip: Be sure to include the mobile version of your online shop in your A/B testing. Some preferences vary depending on whether a customer visits the mobile or desktop version of your site. (not visible when user scrolls down) (always visible/scrolls along with user) (like Version A, but shows at the top when user scrolls up)
Site navigation Test how easy it is to navigate your site and how the elements appear. Ideally, you’ll start with quantitative analyses to define which menu items your users click on the most. Rarely clicked on items should be placed last or removed entirely to keep the list of options shorter and easier to read Original number and sequence of menu items Menu items arranged in order of how often they are clicked Fewer menu items shown, to ensure better readability (alternatively, use a drop-down menu)
Banners It’s important to clearly formulate and showcase the benefits of your products. A/B testing helps you determine whether displaying product benefits in a header or in a colourful banner has a stronger impact on conversion Listing advantages in the product description Showcasing advantages in a colourful banner Presenting advantages in a banner with other colour combinations

Start page

The start page is generally the first page of your website that users will see. To make sure that the customer journey doesn’t end here, it’s important that customers immediately feel welcome.

  • They should feel certain that this site is the right place for them to be.

  • And they should feel enticed to continue on to viewing other pages on your site.

You can increase page view times by including visual elements, especially photos, graphics and special offers. Performing A/B tests on your start page helps you figure out which images have the greatest impact on your target groups. One option is to use the gaze-cueing effect: visitors to your site naturally tend to look in the same direction as the people pictured on your site are looking. This helps you to direct the visitor’s attention to important elements on the page.

A/B testing tips for online shops: Where do I begin? - Homepage | Mollie

Category page

A category page in an online shop provides an overview of the types of products you offer. Here, your shoppers can filter and arrange the products as they wish. An A/B test helps you check whether your target groups have any specific preferences when it comes to how you list the products; for example, do they prefer to see products in a grid with three columns or five?

In general, you should allow your visitors to sort products in various orders, like these:

  • Newest items first

  • Best-selling items first

  • Lowest/highest price first

It’s important to remember the ‘anchor effect’. This means that visitors are influenced by the first price that they notice on your site. They’ll always instinctively compare all the other prices they see on your site with the first one that they saw. So, choose a higher-priced item as the first one your visitors will see (for example, by using ‘Best result’ or ‘Recommended’ as the default setting for how the items are sorted). A/B testing reveals how various factors influence conversion in your online shop.

  1. If your customers see a higher-priced item first and then notice that the next ones are lower in price, they will perceive these latter items to be a better bargain. This makes them more likely to place an order.

  2. However, in some industries, showing a higher price at the beginning of the customer journey can have the opposite effect: it can scare potential customers away, which results in a higher bounce rate.

Product pages

Product pages contain all the important information about the product:

  • Product name

  • Product images

  • Description of features, materials, etc.

  • Price

The goal of these pages is to entice your customers to make a purchase. This table shows how to improve your product pages using A/B testing (scroll to the right to see the full table):

Page element A/B test Version A Version B
Images Increase images sizes Open photos in a new tab when the user clicks on them Increase image size when the user hovers over the image
Position of the image carousel Below the main image Beside the main image
Appearance of the image carousel All available images are visible in smaller/thumbnail form to begin with Only the images before and after the main image are visible in smaller/thumbnail form
Product description Appearance of the product description List of product details can be expanded or collapsed List of product details is completely visible
Recommendation box Position Directly below the product description At the bottom of the page
Wording ‘You might also like ...’ ‘Other shoppers also bought ...’

Checkout

If a visitor makes it to checkout, they’re just a few clicks away from completing an order. To prevent them from clicking away from your shop, use A/B testing to make sure your checkout process is as smoothly designed as possible. Some important topics for A/B testing during checkout are:

  • Voucher code fields: Some shops display their voucher code fields more prominently than others during checkout. If users have to hunt too long to find the voucher code field, they may become annoyed and leave your site. In other cases, if a user sees a voucher code field prominently displayed, it might encourage them to navigate away from your site to search for discount codes. And if they can’t find any codes, they may not return to complete the purchase.

  • Trust: If a potential customer loses trust in your shop during the checkout process, they will most likely leave without completing a purchase. That’s why it’s important to include elements that build trust, such as seals of approval, transparent shipping prices and returns information. You can use A/B testing to determine the ideal size, position and sequence to use for displaying items like these. 

  • Forms: You can test checkout forms to ensure that they are user friendly and not too long. If it takes too long to fill out a form, or if the form requires information that isn’t easy to answer, the user may become frustrated and leave. Try testing to see whether one-page checkout is more effective than multi-page checkout, or whether your customers prefer using auto-fill to fill out forms.

A/B testing tips for online shops: Where do I begin? - Checkout | Mollie

Which A/B testing tools are available?

There are many A/B testing tools that you can use to optimise your online shop’s conversion rate. Let’s look at the three most common tools, along with their features and costs:

A/B testing tool Features Costs
Google Optimize A/B or A/B/n tests Free
Testing for multiple variables Part of Google Analytics
Redirect tests
Optimizely Performance testing for multiple design variations Paid subscription
Testing dynamic websites
Visual Website Optimizer What you see is what you get (WYSIWYG) editor Free 30-day trial version
Heat maps After that, paid subscription
Click maps

What not to do: 5 common A/B testing mistakes to avoid

Now, let’s look at the five most common mistakes that you’ll want to avoid when performing A/B testing on your online shop.

Mistake #1: Focusing only on sales conversion rate

You should be careful not to judge the success of your A/B test simply in terms of sales figures. Here are some other goals that are also relevant when optimising the overall performance of your online shop:

  • Increase in order volume

  • Reduction in bounce rate

  • Increased traffic

  • Visits to the shopping cart

  • Visits to checkout

  • Higher revisit rate

  • Longer page views

These can help you identify further possible areas for optimisation; for example, when there’s a disproportionate increase in two different criteria.

Example: The A/B test results in a 50% increase in the number of orders completed, but only a 30% increase in the number of visits to checkout. This might indicate that your checkout process needs to be further optimised to increase the conversion rate.

Mistake #2: Analysing results too early

If an A/B testing tool reveals that version A is 75% more likely to be effective than version B, then most eCommerce retailers would assume that the A/B test was a success and the results are clear. However, it’s important to keep the test running until it includes an adequate number of users, spread out over a longer period of time. A number like 75% is only significant if the test includes enough people. Other factors influence user behaviour as well, such as these:

  • Season of the year

  • Day of the week

  • Time of day

  • Holidays

  • Events

Check whether factors like these may be influencing the results of your A/B testing. When in doubt, continue testing for a longer period. For a test result to be valid, it should point to a 90 to 95% likelihood that one version is more effective than the other. This is the only way to be sure that it will be worthwhile to apply changes to your entire website.

Mistake #3: Only optimising one page

Conversion is usually the result of a number of interactions that a user has had with your site. In other words, a user rarely visits just one product site and decides to make a purchase without looking any further. They usually navigate to other pages, such as your start page, category page, product pages, upselling and cross-selling offers, shopping cart and checkout. All these steps affect your potential customer’s overall experience. That’s why you should not perform A/B testing on one page, but instead present the user with a consistent experience throughout their entire visit to your site.

Mistake #4: Confusing Google with double content

There are two other things to keep in mind during your A/B testing:

  1. set the alternative version of your landing page to ‘noindex’.

  2. add the canonical tag to the page’s HTML code.

Otherwise, Google will view versions A and B as two pages with identical content (duplicate content). At least one of the pages will then be removed from the index, which harms your SEO ranking. To avoid this, use canonical link elements in your URLs.

Mistake #5: Viewing A/B test results as permanent

If one version outperforms another during A/B testing, that doesn’t necessarily mean it will still perform better six months from now. Remember that other factors, like seasonality, also influence user behaviour. You might find that the conversion funnel changes again down the road, and then it will be time to re-evaluate past findings.

Conclusion: Optimising conversion with A/B testing

As you can see, there are plenty of opportunities for using A/B or split testing to optimise your online shop. Remember to start by carefully analysing your target group and determining where the greatest potential lies for optimising your website. Then set some priorities and start making adjustments. Use A/B testing to see how these adjustments affect your conversion rate. Lastly, remember that these elements generally have the strongest influence on conversion:

  • Recurring elements like headers and menus

  • Category and product pages

  • Checkout

Improve your conversion rate today
Contact us