Statistical and Financial Considerations in Website Optimization
How do I select a good conversion page?
In general, you should choose a page where users complete a defined action that produces desirable business results for you.
If your site has a lot of products and you're trying to test overall conversion improvements, you might want to use a 'Thank You' page as your conversion page - this will enable you to capture any successful action users take. However, if you're trying to test completion of a unique goal, you can narrow your focus with a conversion page that's unique to a specific product - for example, the purchase page for that product.
That's why it's so important to find, and fix, high-traffic landing pages that lose a high percentage of visitors. Look at the "Top Landing Pages" report (See Figure 3) within the Content section of Google Analytics (GA). Pages that have both a high Bounce Rate (the percentage of visits that resulted in the visitor immediately leaving the site) and large number of Entrances need to be redesigned.Note: There is a close relationship between Google Website Optimizer and Google Analytics - the conversion data used in Website Optimizer reports comes from the Analytics database system.
Click here for larger image
Figure 3. The Google Analytics Landing Pages report
The GA Landing Pages report (Content Section) shows a list of top landing pages ordered by the number of entrances on the left. On the right, the Bounce Rate compared to site average is graphically displayed. Pages with a high number of Entrances and a high Bounce Rate (red bar), are good candidates for optimization.
Don't forget about funnel pages
Other high value pages are those that lead visitors to your goal page(s) (See Figure 1). Visitors reach a goal page once they have made a purchase or completed another desired action, such as a registration or download. In GA, you can specify up to ten pages in a defined funnel representing the path that you expect visitors to take on their way to the goal page (conversion). A page that is part of a goal funnel is another great place to focus website optimization efforts.
The Funnel Visualization report within the Goals section of GA shows you how many visitors exit the funnel at each step in the path towards the goal page. In the funnel visualization below, you can see that most visitors in this funnel are lost in the transition from the "View Shopping Cart" step to the "Login" step. Only 7% of visitors move past this step, but of those who do, many go on to complete an order! Before you setup your Website Optimizer experiment, you can examine the Funnel Visualization report to see whether you could be improved simply by limiting steps in paths to a goal, like the "View Shopping Cart" step below.
Keep in mind that sometimes the further the goal is from your tested page, the more traffic you will need.
Click here for larger image
Figure 4. The Funnel Visualization report in Google Analytics
Website Optimizer uses two types of testing: multivariate and A/B split testing.
Two kinds of Experiments
Multivariate tests, the primary focus of this article, allow you to test multiple variables -- in this case, sections of a page -- simultaneously. For example, you could identify the headline, image, and promo text as parts of your page you'd like to improve, and try out three different versions of each one. Website Optimizer would then show users different combinations of those versions (let's say, Headline #2, Image #3, and Promo Text #1) to see what users respond to best. Multivariate tests are more complicated and typically require higher page traffic than A/B tests.
Figure 6. A/B Experiment
An A/B experiment, on the other hand, allows you to test the performance of two (or more!) entirely different versions of a page. You can change the content of a page, alter the look and feel, or move around the layout of your alternate pages; there's plenty of design freedom with A/B testing. It's the simpler type of test, and works best with pages that don't get a lot of traffic.
Two kinds of reports
In multivariate testing, there are two kinds of reports: a combination report and a page section report. Each of the columns in these reports provides a different insight into the performance of combinations, page sections and variations.
The Combination Report
A combination report will show the performance results for all of the page combinations made from the page section variations you created for your experiment. By seeing how well a particular combination performs in comparison with the original and the other combinations, you can choose the most successful one to improve your business.
Click here for larger image
Figure 7. Combination Report in Website Optimizer
Estimated conversion rate range provides the most immediate insight into overall performance. You'll view this column to see how well each combination is performing relative to your original content. If you're a numbers type, you can view the numerical range to the left, but there is a visual display of performance in the bars to the right: bars that veer toward the left and are colored in red aren't performing as well as your original content, while bars that veer toward the right and are partially green are performing better than your original content.
The chance to beat original column shows the likelihood, expressed as a probability, that a particular combination will be more successful than your original content. It is very possible that there can be more than one combination which has a good chance to beat the original. When this number goes above 95% or below 5%, the corresponding bar will be all green or all red, respectively.
Observed improvement displays the percent improvement over the original combination. Because this percentage is a ratio of the conversion rate of a combination to the conversion rate of the original column, it will often vary widely. You should only concentrate on the observed improvement when a large amount of data has been collected and it can be considered more reliable.
Page 2 of 5