Conversion rate optimization

eCommerce Guide
Conversion rate optimization

In this chapter

Here's how you optimize your site to get more conversions. We cover popups, exit-intent strategies and much more.

Exit Intent Popups

Conversion rate optimization is a very powerful marketing approach that’s not used enough.

The best marketing decisions are made using data and CRO thrives on data.

Improving the conversion of your site is perhaps the most powerful way to increase your bottom line.

exit intent popup strategies for ecommerce

A/B testing

One of the fundamental ways to improve your conversions is by A/B testing your site pages and buttons. You can simply set up a heat map to record your user’s activity and then determine which areas of the site are not used, which need to be removed or made more visible.

Exit-intent technology

By Abbey Claire Dela Cruz from Poptin

The goal of any conversion rate optimization project is to keep the user engaged, at every point of the customer journey. And one of the best places to do that is right before they are about to leave your site.

Cart abandonment is very common in the world of eCommerce.

A site user might leave for any number of reasons - found a better price somewhere else, is only comparison shopping, wants to check out if there’s a deal on Amazon, or they have to go do something else.

Cart abandonment happens when visitors enter your site, browse through the pages, add items to the cart, but then leave without buying anything or leaving any contact information.

One highly effective technique to salvage some of these abandoned carts is to use exit-intent technology, which helps in converting visitors into leads or customers before they totally leave your website.

This is how it works.

Exit-intent technology tracks mouse movements and automatically detects once a visitor leaves the frame of the site. When that happens the user gets hit by an exit pop up packed with enticing content and a product promotion.

The reasons behind why exit-intent technology is an effective conversion technique are pretty simple.

1. According to Baymard Institute, 68% of eCommerce visitors leave your site without purchasing anything. Imagine the huge chunk of potential conversions that are left behind when you don't try to stop them before they leave.

2. Visitors probably exit from your page because they didn’t see anything relevant yet, but with exit pop-ups, you have a chance to offer them something more attractive and it helps them to convert.

3. An exit-intent pop up interrupts the pattern of going through the motions of browsing for products and services. When an eye-catching offer pops up from the page, it convinces the visitor to think twice and have a second look.

Here’s an example.

A robot vacuum brand called iCleblo used exit-intent popups to cut its lead costs by 50%.

Here’s how they did it.

Initially, they ran an Adwords campaign in the 2nd quarter of 2016. For a total budget of $6,600, they acquired a total of 86 leads at an average cost of $76 per lead.

However, in 2017, with the same landing page, they decided to partner with an advanced CRO platform where you can easily create your own pop-ups - Poptin.

The robot brand implemented two exit-intent pop-ups, one was a round-shaped pop-up to capture the user’s attention and another was a timer pop-up to create urgency.

Here is an example of a circular exit-intent popup:

circular popup ecommerce poptin example

And here’s a more enticing one with a timed offer:

timed offer timer popup example from poptin ecommerce business

With the same budget, they successfully acquired over 130 leads at an average lead cost of only $36.

Compare that to their $76 per lead cost with ads.

To make your exit-intent tactics a lot more effective, here are some popular exit pop up tactics to steal.

Top exit-intent pop-up strategies

Offer discounts and freebies, especially for first-time visitors.

Whether it’s for online courses, events, or items, who doesn’t want discounts, right?

Surprising your visitors with offerings they know they won’t have at any other time increases your chances of converting them into leads or customers. Most of the time, pop-ups help them decide whether to check out the page or not.

coffee popup example from poptin ecommerce business

Promote related content. If you’re a blogger, you might want to show a downloadable ebook or a link to a related post to visitors who have read an article on your site.

Upsell. By communicating with a sense of urgency through a countdown timer, you can offer discounted items that they can easily add to their carts and encourage checkout.

Ask for feedback. You can conduct a quick survey and ask them what you missed on the site. This can drive engagement and boost email leads.

There are a lot of ways to achieve better conversion rates, and exit-intent technology has been proven to be one of them.

If you want to implement your own exit-intent pop-ups, check out Poptin.

It has a wide range of advanced tools, smart triggers, targeting options, and many other features that help improve your conversions.

ecommerce marketing popups playbook


No items found.

Conversion rate optimization tools

Maximize your site conversions with a complete CRO tech stack

There are a ton of eCommerce tools out there that improve conversions - social proof notifications, time-bound offers, cross-sell/upsell widgets, and even personalization.

Yet the challenge is getting data and proving that each of these makes a statistically significant improvement to your bottom line.

It’s also hard to integrate them all into a coherent marketing strategy.

What’s the solution?

cro checklist and tool

The trick is to use a comprehensive CRO platform like ConvertCart.

This tool combines all the moving parts and widgets into one dashboard. They also have some advanced data capabilities and dynamic personalization features that help you zero in and provide the best possible experience for each site visitor.

The most powerful feature of the platform is that they get data on every single web user and they use all of this data automatically to power up all the tools on their platform. So your web visitor’s data can be connected with your email marketing, your popups, personalization, and notifications, all in one platform.

That is power at your fingertips.

convert cart features CRO tool ecommerce


No items found.

CRO Audit + Checklist

Conversion rate optimization or CRO can be confusing to some people. There are so many gadgets, widgets, design changes to change that the possibilities are endless.

Where do you start? What do you optimize first?

How do you make sure to focus on the features that have the greatest impact first?

That’s why we’ve created this quick CRO audit + checklist.

ecommerce cro audit checklist

It’s a 5-minute questionnaire to see what you are currently doing on your store and then we display your CRO Score and a checklist of all the features that you need to develop more or improve on.

Our audit tool is just a taste of what you would be able to get from a comprehensive CRO audit, by a conversion rate optimization expert. This tool will get you started on some of the key areas of your site, and show you what you could do by yourself or with your existing marketing team to improve your conversions.

Before we dive into the areas that we cover in our audit, let’s first talk about why you need an audit and what it could do to improve your business.

What is conversion rate optimization?

In internet marketing, and web analytics conversion optimization, or conversion rate optimization is a system for increasing the percentage of visitors to a website that convert into customers, or more generally, take any desired action on a webpage. It is commonly referred to as CRO.

In other words, conversion rate optimization (CRO) is the process of optimizing your site or landing page experience based on website visitor behavior to help improve the probability of the visitor taking desired actions (conversions) on the said page. In today's world, online traffic is highly inconsistent.

What is a CRO audit

A conversion rate optimization (CRO) audit is a full audit of your site, traffic sources, and marketing channels and is used to determine all the various things you could do to improve your conversion rate.

CRO is all about taking the traffic your pages get and figuring out how to improve the percentage of that traffic that converts.

An eCommerce CRO audit usually looks at the following:

  • Your site content
  • Your checkout process
  • Email marketing
  • Popups
  • Shipping / Returns policies
  • Social media retargeting strategies

So, that's what a conversion rate optimization (CRO) audit does. It studies and measures the behavior of your users. It doesn't just pick common practices from a proverbial rack. It sees to it that any optimization changes you make are tailored to your audience.

conversion rate optimization cycle cro audit
Image source

Why every business needs a CRO audit

Knowing what your conversion rate is and working to improve it is super important for any eCommerce business.

Let me give you an example.

Bob is your Facebook ads guy, and he comes to you asking to double the monthly advertising budget from $500 -> to $1000.

You immediately say that’s a great idea, but let me do a CRO audit first!

You go check out our CRO Audit tool, and realize that you don’t currently have any exit popups on the site.

So you buy an exit popup tool ($25) and test it out for 2 weeks.

After 2 weeks, it improves your conversions from 2% to 3%. That’s just a 1% improvement, but that affects all of your marketing channels and not just Facebook ads.

If your store generates $1M in revenue annually, then a 1% increase in conversion would equal $10,000 more sales per year.

The best part is that the popups cost you $25/month.

how to improve your conversions with a CRO audit

And that’s why you need to get a CRO audit. To discover new opportunities or optimize existing elements of your site. To improve your overall conversion rate that will improve every single one of your marketing channels (ads, social media, influencer marketing, SEO, etc.).

Here are the steps to conduct your own CRO audit:

cro ecommerce audit and checklist

Step 1 - Define your Business Goals

The first step is to define your business goals. This goes back to your business’ marketing strategy, which is where you should have outlined this part.

You have to set a specific goal that you want to improve on with your CRO efforts and this will really guide you along the entire process.

For example, if your goal is to increase blog subscribers as opposed to increasing sales then your key metrics will be totally different.

If you are looking at sales you should track ROAS or ROI. If you’re looking to get more leads then you should track the number of sales-qualified leads (SQLs).

Step 2 - Identify your Ideal Customer

The next step is to identify your ideal customer. Who are you trying to attract? Who are the customers that you wish to convert better? It’s important to zero in on converting those customers that will have the greatest impact on your ROI.

So break it down into demographic information:

  • Age
  • Gender
  • Location
  • Income bracket
  • Industry
  • Interests
  • Shopping habits/preferences
  • The device they’re using

Step 3 - Make sure your tech works

Now that you have your goals and have outlined your ideal customer, it’s time to look at your site. Do a quick technical audit of your site, to make sure that everything works.

This means looking at:

Mobile responsiveness

Make sure that your site works for all the common screen sizes out there (web, mobile, tablet) and that it is responsive.

mobile responsive ecommerce website cro audit
Source: 3DCart

Page load speed time

Use Google’s PageSpeed Insights to see how fast your site loads on desktop and mobile devices. This tool will show you all the images that you need to optimize, unused code you should delete, and other technical upgrades you should perform to your site.

target pagespeed insights cro audit

Broken pages/links

Make sure that all of your buttons are clickable and your links work. You can use an SEO tool like Ahrefs to find all of your broken links. You can also use a heatmap tool like HotJar to see if there are any elements on your site that are not visible or clear enough for your site visitors.

hotjar heatmap example cro

Step 4 - Look at your content

Now, that you have done a thorough technical site audit, it’s time to look at your content. It’s important that you have something called “message match”, which means that your landing page content matches your brand messaging on social media, advertising, and whatever other marketing channels you’re using.

Personalize your content

Online shopping is all about impulsive decisions it’s a fairly emotional process. Sometimes the user decides to buy a product based on how familiar they feel on the site, or how unique and personalized their shopping experience is.

In fact, studies show that 49% of buyers make impulsive purchases after receiving a personalized experience. So make sure that you use a tool like Google Optimize or Proof to dynamically display different content based on the user’s demographic data - their gender, age, location, interests, and the like.

useproof the personalized content tool for ecommerce

Display product reviews

Word of mouth is the most powerful marketing tactic by far and eCommerce brands can harness its power by displaying product reviews. You can do a ton of things with your reviews.

  • Display your reviews on your product and category pages
  • Embed product reviews into your emails
  • Share product reviews on your social media

Studies show that 84% of people trust online reviews as much as product recommendations from their friends - now that’s powerful.

You can use a tool like or Bazaarvoice to collect and display product reviews from your customers.

bazaarvoice reviews tool for ecommerce brands cro audit

Display user-generated content (UGC)

What’s even more powerful is when you display image or video reviews from your customers, and that’s called user-generated content or UGC. You can use tools like Pixlee or Loox to collect these reviews on social media by scanning for a hashtag or a mention of your brand and to display them on your site or email marketing campaigns.

loox ugc product reviews cro tool

Step 5 - Conduct a design and layout audit

Make sure that your design and layout make sense. This ties back to mobile responsiveness as your layout will change based on the device that your site visitor is using.

Here are some general guidelines to follow:

  • Make sure that your CTAs are clearly visible
  • Use color theory to decide on the right colors to use
  • Don’t be too aggressive in your offers
  • Make sure that your design matches your brand

Here’s a great guide to using the right colors for your site:

color theory emotion cro conversion rate optimization design

Step 6 - Conduct a user experience audit

The most important aspect of your CRO audit should be a thorough look at the way your user experiences your site. This is where everything culminates.

User experience is perhaps one of the biggest factors in eCommerce CRO. There are hundreds of tools out there, for every kind of platform, that can help you improve your users’ experience.

Social Proof

These are messages that pop out on the bottom left corner of your screen and say something like “Sally just bought this product”. Surprisingly enough, they are super powerful. They help the user identify with the site, and can be personalized by their location. So if you’re in Los Angeles, you’re going to see “Sally from LA just bought this product”.

social proof notification tool for ecommerce cro audit
Source: Proof

Urgency & Scarcity Notifications

Another version of this is an urgency notification. This is a notification that displays the number of items left or how many visitors have seen the item in the last 24 hours.

Word of caution: A lot of times brands puff up their stats to make their products look more popular, and that looks fake, so don’t do it. Stay authentic and true to yourself and your brand.

scarcity urgency notification for ecommerce brands to increase conversions
Source: Proof

User identification is super powerful and the more familiar you can make the entire experience to your users, the more chances you have of turning them into happy customers.

Dynamic content

Another powerful way to personalize your site experience for each user is by dynamically changing the content that’s displayed for each user. You can use Google Website Optimizer or a more upscale tool like Proof to change your site’s content depending on the user’s demographics.

For example, if a user is visiting your site from California, you could display something relevant to them (mention the weather or mention your other customers from the same area).

optimizely useproof site dynamic content personalization
Source: Proof

Use smart product recommendations

Product recommendations are another great way to personalize the user experience. A great example of this is Amazon. Amazon has worked tirelessly to perfect the art of displaying the right products for each user.

Look at the example below.


It has the name of the user in the top left corner, a product they recently viewed, a relevant ad, a Cyber Monday deal, and several of the top categories that they are most likely to purchase from.

Every large eCommerce platform has numerous plugins you could use to display personalized product recommendations.

If you’re on Shopify, you could use a plugin like Wiser or Innonic.

smart product recommendations wiser shopify

FAQ pages

This is another way to improve your conversions. Remember, the customers that already know what they want, are going to use the search feature on your site, and are going to be asking the “buying questions”.

Display all the most commonly asked questions about each product on the product page. You can even include an FAQ widget on other pages of the site as well, or make a separate page for it.

You can use a tool like Intercom to add your FAQ’s to a chatbot and bring your customer service to a whole new level.

intercom knowledge base FAQ cro audit

Our Story Page

A recent study by KoMarketing showed that the first thing that 52% of consumers want to see on your site is the “about us” page. And there’s a whole science to creating the perfect “about us” or “our story” page.

It’s a perfect opportunity to showcase your team, brand values, successes, and make yourself or your brand more relatable to your site visitor.

about us page example to increase conversions
Source: Adham

Check out this in-depth about us page guide to learn all the ins and outs of about pages.

Clearance pages

One really important feature of any eCommerce business is - promotions.

Customers expect to get coupons, deals, and clearance sales, especially around the Holidays. Studies show that about 93% of consumers expect a discount throughout the year, so make it easier for them to purchase by displaying all of your promotions and discounts on a clearance page.

Here’s a great example of a clearance page from Target:

target clearance page

Step 7 - Optimize your checkout process

Last but not least, let’s talk about the checkout area on your site. Your checkout process is where the magic happens, it’s where leads become customers. So make sure that it is optimized, easy to follow, and remove anything that’s unclear or that stops people from converting.

Here are a few high-level tips for improving your checkout process.

Offer free shipping

About 79% of all US consumers say that free shipping is a major contributing factor to their decision to purchase a product. Free shipping has become the norm nowadays, especially because of Amazon Prime.

So if you can, try to offer free shipping on all of your products. Increase your prices a bit to keep your margins the same, and you could still charge for expedited shipping.

free shipping banner cro audit

Display shipping at the beginning of the checkout process

One of the worst things you can do is to offer a really good price on a product, and then charge a really high shipping rate, right at the end of your checkout process. Studies show that 21% of online shoppers abandoned their carts because of this exact thing.

So don’t do it.

If you can, offer free shipping on all of your products.

Otherwise, display an estimated shipping charge instead, and make it higher than your average shipping rate in order to underpromise and overdeliver. Then, the customer will be relieved to find that the shipping rate is less than what they thought.

Etsy does a really great job at this, check this out.

shipping rate ecommerce cro audit

Offer Guest Checkout options

Normally, eCommerce sites require a user to log in or register on the site. A guest checkout option is when a user can just put in their information and pay for the product, without having to register and open an account.

This is another barrier of entry for a customer, so make it simple for them to checkout and allow them to purchase your product as a guest.

Here’s another example from Etsy, of a flawless guest checkout experience:

Etsy guest checkout example cro

Offer 3rd party checkout options

Imagine a customer wants to use a PayPal account to pay for your product. They go on your site, they fall in love with the product, they’re checking out, and BOOM.

They found out that you don’t accept PayPal.

What if it’s their only available payment method? Their husband took the credit card that day by mistake? Their dog chewed their debit card?

That’s why you have to have 3rd party checkout options available. That means GooglePay, ApplePay, AmazonPay, PayPal, etc. Connect to as many of these as possible so you could maximize your customer’s ability to pay you.

applepay banner 3rd party checkout payment options ecommerce

Display trust/secure banners

People want to trust the site that they are buying from. That’s a given. There are a lot of ways to show your customers that they could trust you.

One of the easiest ways is to get an SSL certificate for your site. That’s for the notification on the URL field of your browser that shows that it's a secure site.

Also, display some nice banners on your product and checkout pages.

And finally, something super effective is to get shipping and handling insurance like Route.

Route shipping insurance for ecommerce stores

Those are some of the basics of optimizing your checkout section.

Next up - email marketing.

Step 8 - Automate your email marketing

No CRO audit would be complete without a thorough look at email marketing.

Email marketing is a HUGE part of your conversions. Email is still the number one medium that consumers use to communicate with a brand, and that’s true for every stage of the buyer’s journey.

Here are all the elements that you need to have in place to maximize your conversions from email marketing.

Cart abandonment

Cart abandonment flows are the most powerful aspect of email marketing for eCommerce brands. These are the flows that usually make the most money.

Make sure that you are sending every user that abandoned their cart at least 3-4 emails reminding them to come back and purchase. Don’t be pushy, use humor, and spread the emails out from 30 minutes - 7 days after purchase.

Here’s a cheeky cart abandonment email from Casper:

casper cart abandonment email flow

Browser abandonment

The next email flow that you must have is called the browser abandonment flow. This is where a user hasn’t even added anything to their cart yet, they just came to your site and then left.

The only way that you could retarget them is if you have their information. This means they could have bought something previously or joined your email list, or another way that they have left their data-print.

This flow will generate less money for you than the cart abandonment flow but may still bring in a good chunk of change.

Here’s a great example of a browser abandonment flow from Omnisend.

browser abandonment flow

Email your popup subscribers

It’s not enough to display a popup, get the emails of your site visitors, and then simply add them to your list of weekly newsletter recipients.

That’s a waste of a popup.

The real magic happens when you make a special email flow for each type of popup, and send 2-3 email follow-ups to each lead.

In general, open rates are 50% for these, so they are just as powerful as cart abandonment emails.

If you’re using a popup tool like Sumo, you can easily add automatic email responses and follow-ups for each popup you create.

automatic email responder from sumo popups cro audit

Step 9 - Streamline your customer service

Yes, a CRO audit is mostly about marketing, the customer service team is just as much of an influence on your site conversions as the marketing team. This is exactly what the top brands have realized and what made them successful.

The customer is #1 and your customer service policies have to reflect that.

Let’s dive into a few high-level tips that will help you improve your customer service and increase your conversion rate.

Improve your return policy

Easy, no-hassle, no-questions-asked returns are the standard. This is what customers get when they shop on Amazon and this is certainly what they expect when they go on your site.

So provide it.

Make your return policy super easy to follow, pay for each return (if it’s feasible), and display free return banners on your site to improve customer’s trust in your brand.

free return policy southern tide conversions
Source: SouternTide

Improve your response time

Customers love to get answers and be serviced within a day. The longer it takes the more chances that they will choose to shop somewhere else. Utilize chatbots and automated knowledge bases to answer as many questions as possible on the spot.

Then, work on improving your customer service team’s response time to every customer ticket that comes in. That might mean having someone man the phones/inboxes on the weekends or after hours.

Studies show that chatbots help brands decrease their average response times from 24 hours down to 10 minutes. Here’s some data from Tidio that shows that for eCommerce almost 50% of conversions happen in that 10-minute window.


Improve your delivery time

Delivery is key.

Your customer wants to get their product fast, and this is especially true around the Holidays, which is the busiest time of the year for any eCommerce shop. You can use a service like Deliverr to help you get that delivery time down to 1 or 2 days.

That means you can be as quick as Amazon, which is a HUGE deal for any brand.

deliverr 2 day and next day delivery cro

Make it easy for customers to contact you / contact us page

Finally, make it easy for your customers to contact you. Make your contact page easy to find by putting it in the header and the footer of your site. Display your phone number, email, and link to your FAQs page or knowledge base.

Here’s a great guide from Shopify on optimizing contact us pages.

And here’s a great example from Bando:

bando contact us page example

Step 10 - Use popups

We’ve saved the best for last.


Popups are a super powerful way to increase your conversions. Now, that doesn't mean that you should blast your customer with every popup imaginable.

Popups should be used in an organic way, should fit your brand and your customer persona.

Here’s an example of a popup experience gone wrong.

In this example, I was hit with 3 popups all at once - a regular popup, a chat popup, and a push notification.

How not to use popups

This is a nightmare of every UI/UX designer. Our designer is still in therapy for it (🙃).

In reality, the brand may not be doing this intentionally. On their own, these 3 channels are perfectly great for getting leads and improving conversions. But the brand either didn’t realize that they popup out at once, or didn’t set them up properly.

In this case, the best thing to do is to display only one of these popups for each new user. Because when you blast them with three at once, their reaction is just like mine was - I’m out of here.

So let’s go through each type of popup.

Use exit popups

Exit popups are a fantastic way to convert your visitors into leads and paying customers. It’s a relatively noninvasive popup that only displays when the user is about to exit your site.

exit popup example

Use push notifications

Push notifications are another great channel. These are those notifications you get in your browser or your phone. You can use a tool like Subscribers to set it up and they even have some advanced eCommerce features like cart abandonment and welcome flows.

Subscribers push notification tool cro audit

We actually wrote an entire chapter about push notifications in our eCommerce guide. So head out there to learn the ins and outs of setting them up.

Use Messenger / chat popups

Last but not least, a chat popup is by far the most user-experience-friendly popup there is. It provides a great way to start a conversation with your customer. You can use Manychat or Intercom to set one up.

We’re not talking about the ones that pop in the center of your screen, but the ones that pop out of the chat widget on the bottom left or right corner.

Here’s an example from the AI platform Syte:

Notice how organic and non-invasive it is.

That’s one beautiful conversation starter (pun intended).

cro audit tool


There is a lot more to a CRO audit than meets the eye.

Here’s the step-by-step process:

1. Define your business goals

2. Identify your ideal customer

3. Optimize the technical elements of your site

4. Look at your content - UGC, dynamic content

5. Improve your design and layout

6. Optimize your site for the user experience

7. Improve your checkout

8. Automate your email marketing

9. Streamline your customer service

10. Use popups the right way

And this isn’t a fit-all solution.

Every site needs its own mixture of the right CRO elements to really shine.

If you’d like to get a customized CRO audit, check out our CRO audit tool 😉.

The Mayplers.


What is a CRO audit?

A conversion rate optimization (CRO) audit is a full audit of your site, traffic sources, and marketing channels and is used to determine all the various things you could do to improve your conversion rate.

What is CRO process?

CRO is all about taking the traffic your pages get and figuring out how to improve the percentage of that traffic that converts.

What is a conversion audit?

It's a 360-degree assessment of the customer journey on your site. The purpose of the audit is to issues that could be decreasing the conversion rate on your website.

How do you calculate CRO?

To calculate your conversion rate simply take the number of conversions and divide it by the total number of visitors, then multiply by 100 to get a percentage.

How to Run an A/B Test - for eCommerce

With a contribution from Jacob Elbaum, CEO at Shivook.

The main goal of conversion rate optimization is to increase your site conversion, and you can’t do that just by adding a few popups and plugins.

You have to actually change elements on your site. But you can’t do that willy nilly. Some of the worst marketing decisions are made because of an opinion or a “gut feeling” of someone on the team. You have to base every decision on real user data.

ecommerce cro audit checklist

So, before you change something you need to test it first, and that’s what an A/B test is all about. It’s the most powerful technique a marketer has in their arsenal to be able to move the needle in the right direction. With the correct A/B test methodology a brand could improve its design and UX over time to achieve its sales and marketing goals.

In this post, we’re going to teach you how to set up and perfect an A/B test for your eCommerce site.

Let’s jump in.

What is an A/B test?

An A/B test is a process of showing two variants of an element on a site, displaying it to two different segments of the audience, and measuring the conversion of each. On average, an A/B test should run for a minimum of 2 weeks but can go up to 2 months (depending on the amount of traffic a site gets).

What is an A/B test vs a multivariate test?

An A/B test is when you are testing two or more versions of an element on the site. A multivariate test is when you test multiple combinations of a few key elements in order to find the winning combination.

Here’s a visual explanation of the difference from VWO’s guide on A/B testing.

what is the difference between an a/b test and a multivariate test

A multivariate test has more complexity and should only be used for large sites. If your site is too small and doesn’t get more than 20k monthly visitors then you won’t be able to collect enough data for a multivariate test.

You can either pick two combinations that you think are the best ones and test those or test each element in a separate test. It will still give you a similar result, but will just take more time.

When should you A/B test?

Ask any CRO marketer and they will tell you that you should constantly be testing. Every 1-2 months, you should have a different test set up. The problem is, with smaller companies & brands that kind of pace is just not realistic.

Smaller brands often lack the time and resources to put into running an A/B test every month. If you do have the resources, you can greatly improve your bottom line and overall performance of every single marketing channel, with the right testing strategy.  

If you don’t have the resources then you need to save an A/B test for when you really need it. There are different scenarios when you should run an A/B test, let’s explore some of them.

When launching a new feature

This is part of the agile methodology that we talked about before. The idea here is that instead of throwing money at developing a brand new site, you can develop one feature at a time while testing every single change.

So many companies do massive redesigns only to see their conversions flop. So be smart and use the agile way of creating one feature at a time, and testing as you go.

So when you are ready to launch a feature, first create your prototype(s). Then, run an A/B test with your existing version and your new version, and see which one does better.

To increase your bounce rate

Sometimes an entire category is not selling well and your sales guy wants to take it “off the shelves”. This is a great time to look at the specific pages that aren’t converting, and see if you have a problem with the bounce rate.

A bounce rate is the time it takes the user to land on your page and leave. If the bounce rate is really high then your users are leaving without purchasing anything. There could be a number of reasons - too many options to choose from, a message mismatch, or not enough clarity in the CTA’s and various elements of the page.

This is a great way to pause and test the most problematic pages. You could change the colors, the layout, make the CTA’s clearer, or change the content on the page.

To improve a low-converting channel

Sometimes a marketing team launched a channel - say Facebook ads - and they’re not successful. The conversion rate on that one channel lags behind the other already existing marketing channels.

It’s not always a problem with the channel itself, but with the landing page.

Sometimes a simple tweak can bring that message match that the users from that one channel are looking for.

One of the brands I was working with was getting a lot of clicks from Facebook ads but very little sales. They saw that their video ads were doing particularly well so they A/B tested adding more video content to their pages and they started seeing lower bounce rates, longer visit duration times, and more conversions.

ecommerce cro marketing checklist and audit

Focus on the customer

The secret behind every successful A/B test is keeping your focus on the customer.

Businesses that are in touch with their customer’s needs, demands, requests, are much more likely to experience above-average conversion rates.

At the end of the day, it’s all about the elements and the design that helps your customers convert, and not the elements and design that you or your team like best. A business that focuses on the customer will be more successful because it is listening to its customers’ needs and designing its site and user experience accordingly.

Source: SuperOffice CRM

Create a customer-centric system

The Achilles heel of customer-centricity is getting the wrong data or asking the wrong people.

Most people think that the best way to improve is to just ask the customer. Businesses often end up asking the wrong group of people. This is known as a sampling bias - picking the wrong sample.

I worked for a startup a number of years ago where they were creating a new site. They decided to go outside and literally talk to strangers on the street and ask them what they thought of their site.

What ended up happening?

They spent way too much money developing features based on that feedback, took too long to get an MVP (minimum viable product), and had to shut down.

Such a shame, it was a really good idea, just the wrong execution.

So when it comes to gathering data from our actual or potential users, we don’t want to make this a disorganized process, where we randomly reach out to some 10-20 customers, start asking them questions, come up with solutions and then implement them right away.

The cost of something going wrong in this kind of ‘lazy’ process far outweighs the potential benefits.

What’s wrong with this approach?

Simply put, implementing sitewide changes based on a 10-20 person sample size can lead to very poor website performance. There could be lots of biases in the data, and none of it could be accurate. You need to systematically collect user data on your site for an A/B test to work properly.

For that reason, it’s important that we have an internal process for conversion rate optimization efforts, a step-by-step framework for improving the conversion rate on the site.

Use a step-by-step method to improve your conversions

Here’s a super simple CRO method that was developed by one of our good friends Jacob Elbaum, who is the CEO of an eCommerce CRO agency called Shivook. You can adopt this methodology to your own website and tweak it as you see fit for your exact needs.

Jacob has also put together a useful worksheet for our readers.

In fact, he explains that his methodology actually comes from another one (called Experrimento), which is more focused on product management. He saw Experrimento in action while working at Voice123, and tweaked it to fit CRO.

In short, the methodology is as follows:

  • Observe
  • Research
  • Come up with ideas
  • Create a hypothesis
  • Wireframe
  • Design/Prototype
  • Test Usability
  • Document Specification (Process Logic)
  • Conduct the A/B test
  • Analyze
  • Implement

Throughout the rest of the post, I am going to dive a bit deeper into the different steps of the methodology so that you can use this post as an optimization checklist for your efforts.


Observe is the first stage of the prices. This is where we gather all of our user data. For most sites, it starts by looking at the Google Analytics data in order to answer one simple question.

Where can we have the greatest impact on conversion rate right now?

In order to discover this, we first must be 100% sure that conversion/goal tracking is set up 100% correctly in Google Analytics. If not, we’ll be wasting time trying to improve something we cannot track.

google analytics ecommerce cro a/b testing data
Image source: Seer Interactive

To answer the “where”, we must look at the data, which will hopefully show us specific user segments where the conversion rate is relatively low or relatively high.

Then, we can dig deeper into those segments to find the micro-segments which are underperforming and dragging conversion rates down.


Research is the combination of a few elements which help us answer the question ‘Why’; that is, “Why are people behaving like that and dragging down conversion rate for that microsegment?”.

Our research most commonly includes the following 2 steps:

Step 1: Surveys

Set up User Surveys on the Key Pages Identified from the ‘Observe’ Stage.

Depending on the page & question you may want to use exit-intent popups so that you don’t add unnecessary friction to the user’s experience.

You can get user feedback through a handful of tools.

If you use a tool like HotJar, you can use their “Question Bank” to come up with question ideas.

hotjar usability test question bank
Source: HotJar

The goal is to identify and discover why people are not converting on the site, and what are some of the issues that contribute to that.

Step 2: Heatmap/Scrollmap Setup & Analysis

Set up heatmaps on those key pages identified in the ‘Observe’ stage & once enough data is collected, analyze the results.

You should be looking through heatmaps, scroll maps, and ideally reviewing some session recordings as well.

heatmap ecommerce data CRO a/b testing

Once you’ve analyzed user feedback and combined that information with user behavior analysis (from your heatmapping tool), you can document your research & look for common patterns of conversion bottlenecks.

Come up with ideas

Now it’s time to take the research from the previous stages along with inspiration from other brand experiences and come up with an idea for what we’re going to be testing.

This is often explained as the potential ‘solution’ to the problem.

Before coming up with ideas, you may want to spend a few hours to complete an internal CRO audit. This way you will know that all of your

Come up with a hypothesis

A hypothesis is an educated guess about what should be done with the data we got in the first stages. Pick one or a few of your top ideas and really develop them in this step.

When we create a hypothesis, we are really doing two things.

Firstly, we’re documenting a benchmark by which we’d like to improve our conversion rate.

It’s best to get as granular as possible here in the data so that we can measure fluctuations in the segment conversion rate more effectively later on.

Next, based on past A/B test learnings and/or the amount of user feedback we have identifying the problem we need to solve, we can come up with a general estimation of how much we’d like to improve the conversion rate in relation to its benchmark.


Once we’ve come up with some initial ideas, we’re going to want to build a more visual representation of those ideas by creating a wireframe.

mobile web wireframe tool a/b test for conversion rate optimization
Source: Balsamiq

The wireframe is a rough skeleton of how we see the idea being implemented on the website.

It is used by the designer to get a general understanding of element placements.

Create a design/prototype

Now take the wireframes that you’ve made and bring them to your design team. It’s time to build on them and turn them into real designs for your site or mobile.

wireframe moqups tool for cro data a/b testing
Source: Moqups

Test the Usability

Double and triple check every prototype you made because you don’t want to be running A/B tests for solutions that have obvious usability issues.

A usability issue is best described as something which does not work as one would expect. If enough people encounter the same expectation and the solution does not meet that expectation, it is unlikely to serve as the solution to the initial problem.

userbob cro tool usability testing

Usability testing can most easily be done via 3rd party websites like,, and a handful of others.

After doing usability testing, we document feedback from the testers so that we can develop A/B testing variations that will perform AT LEAST as a minimum viable product (MVP).

Document Specification (Process Logic)

Documenting a specification (process logic) is where you explain how the design variations for the A/B test should function.

Why is this important?

You may have sections of the design which are dynamic based on some specific criteria. It’s important to use the specification to help explain to the developers what those criteria are and how they will operate and affect the live design.

This is where you will be writing out exactly what changes you need to be made and how everything works.

Conduct the A/B Test

Although this post is primarily focused on A/B testing methodology, this is actually the only part of the methodology where we’re performing actual testing.

That’s because, as explained above, improving conversion rates is more about a methodology than just testing ideas.

For A/B testing, it is highly recommended to use an A/B testing tool to set up your different variations, once they’ve been developed by your developers.

Google Optimize logo a/b test

Here are some recommended A/B testing tools:

Here’s a full list of A/B testing tools.

Analyze the results

When the A/B test is finished, one of the most important (and often overlooked) stages of the testing methodology is analysis.

At this stage, it’s vital to review the data and try to gather as much information as possible regarding user behavior across the different testing variants.

google optimize results screenshot a/b testing cro ecommerce

Variants that greatly overperformed are equally interesting to analyze as those that greatly underperformed.


Because this reveals a lot of insight about the customers and how they are using our website. It helps you determine which elements on your site influencer purchasing behavior the most.

The analysis is usually done through Google Analytics & Heatmap/Scrollmap/Session Recording tools.

Make sure to document your analysis to help guide you in future testing.

Also, make sure that your test results are statistically significant. You can use a simple statistical significance calculator to find that out. This will show you which results are influential and which ones you should disregard.

a/b split test significance calculator testing cro ecommerce


When a test shows statistical significance and in a positive direction, it’s time to implement the test changes across the entire website.

This may require some optimization of the copy and/or the design, and may also require some additional development work that was not done on the test version.

This is something to consider when moving from the testing phase to the implementation phase.

Remember, the main purpose for any A/B test is to improve site conversions and make more money. If your A/B test is successful, it should yield a positive result. Anything less than that requires more testing.

Here’s an example of a successful campaign, and something that you should be striving for:

optimizely a/b testing ecommerce
cro ecommerce marketing checklist


We covered a really good and comprehensive method to set up and run an A/B test for your site. If you follow this process you are guaranteed a long-term significant impact on conversion rate.

All else is a game of risk and taking chances - something less recommended in the world of business, unless you have the resources to do so.

You can use the methodology outlined above or tweak it and create your own. And remember, always make marketing decisions based on actual user data.


Jacob & the Mayplers.


What is an A/B test?

An A/B test is a process of showing two variants of an element on a site, displaying it to two different segments of the audience, and measuring the conversion of each. On average, an A/B test should run for a minimum of 2 weeks but can go up to 2 months (depending on the amount of traffic a site gets).

What is a conversion audit?

It's a 360-degree assessment of the customer journey on your site. The purpose of the audit is to issues that could be decreasing the conversion rate on your website.

What is CRO process?

CRO is all about taking the traffic your pages get and figuring out how to improve the percentage of that traffic that converts.

How do you calculate CRO?

To calculate your conversion rate simply take the number of conversions and divide it by the total number of visitors, then multiply by 100 to get a percentage.

What is a CRO audit?

A conversion rate optimization (CRO) audit is a full audit of your site, traffic sources, and marketing channels and is used to determine all the various things you could do to improve your conversion rate.

Get INSTANT Access to World-Class Marketing Experts.

get started
check icon
Monthly subscription, no commitment