Low Data Rate A-B Split Case Study – PowerOptions – 75% Increase in Sales

SiteTuners is known for its proprietary TuningEngine^SM technology (based on non-parametric tuning) for running very large conversion tests (with over 1,000,000 “recipes” or distinct versions of a website). The TuningEngine^SM also takes variable interactions into account. The combination of large test sizes and considering variable interactions reliably produces the best possible results.

So we are often asked “Why wouldn’t I always use the TuningEngine^SM for my tests?”

The answer is that the TuningEngine^SM is like a high-performance race car. It won’t even work without having a high enough website traffic flow for the test. We recommend a bare minimum of 100 conversion actions per day, and many of our clients have orders of magnitude larger data rates. A conversion action is not the same as a website visitor. We don’t really care about how many unique visitors you have – we need to count the actual actions that they are taking on the site. Actions include downloads, form-fills, or purchases.

So what if your data rate is lower?

Then you are back to doing A-B Split and Multivariate / Taguchi testing. There is a minimum data rate required even for these simpler testing techniques. We recommend at the very least 10 conversion actions per day for A-B Split testing.

There are two approaches to A-B Split testing:

1. Change one page element at a time (e.g. headline)
2. Redesign the whole page and see which version is best

For our client PowerOptions we chose the later approach. There were too many fundamental problems with their free-trial sign-up process to tinker around the edges. We put our best thinking into it and came up with two completely different alternative designs. Each design was a radical simplification of their sign-up pages. One was a two page version (to match their original approach). The other was a one page version.

The data rate was a very low 10 conversions per day. But because of the radically better performance of our new designs, the test was completed in a reasonable time frame. The company tracked the free trial sign-ups to actual eventual paid clients and found a stunning increase of 75% in sales.

See the PowerOptions case study.

The moral of the story is: You can get solid results even with A-B Split or Multivariate / Taguchi testing when you have low data rates. But if you have a high data rate, the TuningEngine^SM will produce the best results.

See a comparison of tuning methods.

Landing Page Optimization Book

Landing Page Optimization is a hot topic. But most of the coverage has been in the form of short online posts. Some full length books exist but they mainly talk about good web design and usability. This is fine as far as it goes. But no “expert” is right all of the time. In fact I am often stunned at the designs that people prefer. They certainly would not be the ones that I would like best.

The real experts are the visitors to your website or landing page. The best way to incorporate their wisdom into your website is through scientific testing that increases conversion rates and the value that you extract from visitor stream. I am happy to announce that I have been chosen by John Wiley Publishing to write a comprehensive book on this subject.

Landing Page Optimization:

The Definitive Guide to Testing and Tuning for Conversions.

It should be available later this year. In the meantime we will be making extended excerpts available as free whitepapers on our site

http://sitetuners.com/downloads.html

and in posts to this blog. For additional information please visit:

http://sitetuners.com/landing-page-optimization-book.html