There’s a famous story in the design world about Google, who tested 41 different shades of blue. It’s often shown as an example of why AB split testing is bad or excessive, and why designers should just be allowed to just do their own thing (that’s what they get paid for, right?).

But using Google’s obsession to prove that testing is always bad does a disservice to the medium. Testing can be an essential tool for web designers who may occasionally forget that design can’t just be beautiful: it must be functional as well.

Testing 41 shades of blue might seem excessive, but testing blue versus purple link colors will show you exactly why there are some design standards that shouldn’t be altered.

A few examples of when to use AB Testing

Old vs. new: AB testing can be essential in the redesign process. When 37 Signals decided to do a redesign of their Basecamp homepage, a major concern was whether the new design would send more visitors to the signup page. By AB testing the page they received hard proof that the new design was heading in the right direction, at least enough for them to push forward with the redesign.

Show your value to clients: Nowadays many clients are more interested in making more money than what your designs actually look like. Showing your clients the increased conversion rate from your AB testing is a great way to connect dollar signs with your design.

Question conventional wisdom: Sometimes you need to question conventual wisdom. Does red always mean stop and does green always mean go? Not always!

Potential Pitfalls of AB Testing

Define what a conversion is: It is important to define exactly what a conversion is so you can measure your progress. If you want to AB test, this is one of the most important steps. Is a conversion a sign-up? A purchase? Until you pinpoint exactly what a conversion means to you, you won’t be taking full advantage of what AB testing can offer.

Local vs. global maximum: Sometimes people get so focused on testing an existing design that they forget about the rule of the local maximum. Though AB testing small changes can bring up your conversions, at some point you may need an entirely new design to see additional increased conversions. When you reach the glass ceiling (a local maximum), it’s time to think about moving to a new design. Josh Porter has a great blog post on why AB testing is not all about small changes.

Following AB testing blindly: An easy trap to fall into is blindly repeating tests without thinking about why you are changing an aspect of the design. Is it worth your time to test 11 different similar fonts to arrive at the fact that one converts 0.01% higher? What have you really learned?

AB Testing Resources

Performable: The Performable software platform gives you a powerful way to quickly create and test webpages. Easily import existing pages and test them against new designs.

ABtests.com: A great way to share your tests with other people and learn from what other people have tested.

ABtests Facebook community: Learn from AB testing contests which variations come out on top. Every Monday a new test is posted and every Friday we tell you what the answer is (and why it won)!

ABtest Calculator: Using this simple tool you can calculate when you have enough data to finally stop a test – it calculates statistical significance and lets you know which variation to pick.