A/B testing in UX is a quantitative research method of determining which design or content performs best with your intended user base.
Also known as split testing, A/B testing presents two variations of a design to an audience and measures which actions they take (or do not take). Once the testing period is completed, you can then analyze results to determine which variation or elements performs better.
Ultimately, the quantitative data obtained through the A/B testing allows the UX designer to determine the best version of a product and make more informed choices when moving forward.
How A/B Testing Works in the UX Design Process
The first step to A/B testing in UX design is deciding what you want to test, as well as what user behavior you will be monitoring.
For example, you might pick two versions of the same website landing page, one with a banner ad and another with an opt in popup.
During the test, you can monitor conversion rates to see which variation is performing better with your target audience. Once the appropriate sample size has been reached (or when the test results provide enough statistical significance to prove the winning variation), you can then end your test and move forward with analyzing the results.
7 Steps to Running Your Own A/B Testing
Here are the seven steps to follow when conducting A/B testing:
Step 1: Decide what to test
When A/B testing, it's important to be very specific with which elements you want to test. If you're testing a CTA button, for example, you should ensure that the CTA button is the only element on the website page that differs on each test. (If there are multiple different elements on your page, it can be impossible to determine exactly what the users are responding to when they take action).
Step 2: Determine the goal of the test
Are you testing a CTA button to determine which results in higher conversion rates? Or are you looking for data to help you onboard mobile app users more thoroughly and efficiently?
Step 3: Create a hypothesis based on the goal
Now, it's time to write out a hypothesis—or a proposed solution—that you think will best improve the user experience.
For example:
Changing the CTA button from grey to orange will make it stand out more and improve our click through rate.
Step 4: Build out the test
There are many A/B testing tools that you can use, like VWO , that allow you to set up your design versions and automatically present them to your audience.
If you're working with an existing website or product, you will set the existing version as the control, while
Step 5: Conduct the A/B test
When you run the test, your testing tool will automatically ensure that each of your variations is shown 50% of the time. (If you have more than two variations, the test becomes a multivariate test, rather than a simple split test).
The testing period might last several days or weeks, depending on how much time it takes to ensure that each variation has been shown to an appropriate sample size.
Step 6: Analyze the data
Some A/B tests are inconclusive. Other tests have a clear, statistically significant winning variation.
Statistical significance is extremely important, since it ensures that your tests results are accurate, and not a result of random chance or error.
Step 7: Use the data to improve the product
When the final results are in, you can either choose to move forward with the winning design, or go back to the drawing board to test other variants.
Benefits of A/B Testing in UX Design
There are many reasons to conduct A/B tests during the UX design process. A/B testing allows the designer to try out different approaches to the same design and figure out how to move forward with the design process. It demonstrates how even small changes can influence the user’s behavior and boost conversion rates.