For many developers, launching a site is not the end of the design process. To continually improve the success of their design, these developers turn to A/B testing. This relatively simple process can teach you loads about what your users are looking for as well as what they ignore or find unimportant.

Today we’ll take a quick look at what A/B testing is, the benefits of implementing it, and some tools to get you going.

What is A/B Testing?

A/B testing is extremely simple in concept. It’s basically testing the effectiveness of different designs to find the optimum solution. These tests are usually performed on live sites with real users who are completely unaware of the test. To clarify, lets look at an example.

Is Bigger Better?

Let’s say Vanilla Forums wanted to know if their current homepage could be improved by increasing the size of the sign up button. The first thing they would do is create an alternate version of their page containing the desired revisions. The image below shows their current page followed by the page they’d like to test:

As you can see, the new version differs enough to theoretically impact the way users perceive the page. However, in order to get a good idea of the impact of the placement of that single button, we haven’t moved much else on the page. You can of course use this method to test the difference between two completely different designs, but the thing I really want to highlight is that A/B testing provides you with solid information on how to improve a current design without making a complete overhaul of the site.

Now Vanilla Forums would setup their site to show version one to half the visitors for a specified amount of time and version two to the rest. Then they would compare the results of the two pages to see which snagged the greatest amount of users (using some tools we’ll discuss later).

Benefits

I’m sure you can immediately name several of the benefits this method would bring to your own site. Let’s go over a few.

It’s Easy!

First and foremost, moderate A/B testing is neither expensive nor difficult to implement all on your own; no expensive consultants or outside agencies required. It definitely takes some work but fortunately there are some great resources out there to help you do the heavy lifting.

Hard Evidence

Using A/B testing lays to rest any usability arguments you might be having around the office or even within your own mind. If your team is torn over two possible scenarios for your site, you can have the various parties dispute the matter all day long but the bottom line is you simply can’t know for certain what will appeal to your users most without testing it out in the real world. A/B testing provides you with an easy way to prove who is right through actual statistics from user interactions.

Incremental Improvement

I’ll again stress that one of they key pieces of potential here is the ability to incrementally improve the interface and/or messaging on your site. As a site owner, this can provide you with a solid way to improve your earnings over time. As a developer, this could provide you with a way to form long-term contracts with your clients.

Things to Keep in Mind

Despite the straightforward nature of the concept of A/B testing, there are several things you should be aware of and considering as you implement it on your site.

What’s Your Goal?

“Better” is quite the relative term. You can’t know if one page is better than another unless you define the terms of the judgement. Similarly, when performing a test, you can’t run wildly into it without consideration of what it is you want to observe. For instance, the Vanilla Forums example above was meant to increase the number of users that click the button to sign up for the service. Note that this is a clear goal that is easily testable, not simply a vague statement regarding the aesthetic nature of one layout over another.

There are several possible goals and metrics to use when testing the efficacy of one design over another. These include the total time spent on the site, the number of pages visited, the percentage of users who navigated to a specific page, etc.

There Won’t Always Be a Clear Winner

Despite the fact that A/B testing can give you clear proof of the superiority of one design over another, sometimes the result isn’t so clear cut. It’s possible that something that you imagined would play a key role in the actions of the users doesn’t really affect them in any significant way. Further, even if you observe a difference, that doesn’t necessarily mean that the difference was a result of the variable. Basic statistical principles prevent a degree of random fluctuation between the results of each version, the goal is to spot differences beyond what would normally occur even if each of the two groups received the same page. Keep in mind that the more users you have to take part in the test, the greater the accuracy of the results.

Resist the urge to nit-pick every tiny portion of your site and change something every time you see the slightest increase in conversions. Look for significant ways to improve your site through clearly met goals.

Multivariate Testing is Also Possible

If you want to go beyond testing the effect of a single variable, multivariate testing provides a way to change up the mix of elements shown to a given user to track which variables have the greatest impact.

Tools and Resources

Now that we’ve briefly gone over the ins and outs of A/B testing, I’m sure you’re eager to get started. Check out these great resources to get you on your way to enhancing your sites.

Google Website Optimizer

If you’re just getting started with A/B testing, Google Website Optimizer should be your first stop. It’s fairly straightforward and easy to use and has a lot of resources to walk you through the process of testing (A/B or multivariate). Best of all: it’s completely free.

Alternatives to Google

If you’re not impressed with Google’s offerings, check out these other options.

Visual Website Optimizer

SiteSpect

Vertster

ABTests

More Info

Want to know more about A/B testing? Check out these other great articles.

• Fundamentals of A/B and Multivariate Testing (Pro Blog Design)
• How To Increase Site Performance Through A/B Split Testing (UX booth)
• Split A/B Testing (Webcredible)
• How to do A/B Testing in WordPress (Carsonified)
• Ultimate A/B Testing resources and how I used them (Brian Cray)
• An Introduction to Website Split Testing (Six Revisions)

Conclusion

You should now be equipped to begin testing your site for potential improvements. Get started today and let us know in the comments below what section of your site you think A/B testing is most appropriate for.