Statistical and Financial Considerations in Website Optimization
Statistical and Financial Considerations in Website Optimization: Part 1
By Marcia Gulesian
Most comapnies want to deploy Web pages that fulfill a business purpose. Their goal might be selling their product (at a profit!), distributing information (such as newly legislated prerequisites for obtaining a driver license), gathering information (such as the demographics of potential customers) from an on-line form that doesn't turn people away because it's too complicated or intrusive, etc.
The question is what do you put on your page(s) to achieve your goal(s)? You could decide on your own or together with a team of in-house experts and hope for the best. Or you could throw the process of Website design open to the world of users beyond your firewall. In doing so, you would be democratizing the process by letting the public tell you, through their actual behavior, which of a number of possible designs gets them to use your Web page(s) in the way that you had hoped they would. As pointed out in a recent New York Times article, the latter approach has now become main stream.
We will focus primarily on one of the several competing products that you could use to democratize this process, Google's online Website Optimizer (GWO), chosen for, among other reasons, the fact that Google doesn't charge for its use. Moreover, in a future article, I'll discuss the collateral use of some of Google's other online services:
- Webmaster Tools
Also, Urchin, Google's counterpart of its Analytics , that, unlike Analytics, you can install on a local server located behind your firewall.
First, I'll elaborate on Optimizer's use of some elementary statistics in the preparation of its reports. I'll then apply additional statistical methods to the raw data collected by GWO, after exporting this data to Microsoft Excel (to which I have added the statistical tool StatTools from Palisade, Inc.).
GWO allows you to try different layouts, alternate content, new buttons, and even new colors through multivariate testing or split A/B testing without making permanent changes to your site, these tests will be described in some detail below. Better yet, your visitors themselves ultimately determine which combination of site elements cause them to perform the actions you desire such as filling out a form, adding an item to the shopping cart, or taking the big plunge to enter their credit card.One problem with this approach is that this approach modifies the original website and some of these experimental variations may cause some visitors to abandon your site. This may not be acceptable for businesses with low-traffic, high-stake websites and others.
Choose the pages and content to test
Using GWO's Web-based interface, you provide Google with the content -- headlines, images, or text, for example -- and design alternatives that you'd like to test.
Test these changes with your visitors
GWO will then show these content and design alternatives to your site visitors, all the while monitoring which combinations lead to the highest conversion rates (another term that I'll define in some detail below). An outline of this process is shown in Figure 1.
Click here for larger image
Figure 1. Overview of the Website Optimizer process. GWO monitors the conversion rate of alternative Web page designs.
This is a good time to introduce some of the basic definitions used throughout this article:
Sections: elements you test on a page; e.g., headers, images, buttons, forms, and text.
Variations: different ways you can design and word these elements.
Combinations: different ways variations across elements can be matched up.
Conversion page: the page that, when reached by a user, means business results for you. It's the "Goal" in Figure 1. Depending on your type of site, it may be the page where a user will complete a purchase or fill out an interest form.
Conversions aren't always associated with user clicks, page visits, or other actions in the browser. A user who finishes reading an article, for example, won't always click a button or register another event in the browser to indicate that she's done.
In those cases, it may be useful to count a conversion after the page has been loaded in the browser for a certain amount of time. This way, users who stay on the page for 20 seconds (or for any amount of time you set) will trigger a conversion in GWO. We'll cover this subject in a bit.
Conversion rate: the percentage of your visitors that end up reaching a given goal during the time period in question.
When you start testing more than one element on a page at the same time, you're multivariate testing, with its combination of sections and variations.
In GWO's A/B test, you treat the entire page as a section; you are testing pages, not variables. Because there are only two combinations, it is particularly suited for pages with little traffic; you don't need a whole lot of traffic to be able to identify a difference (if there is one).
Click here for larger image
Figure 2. This chart includes your web site, some avenues people use to access it and two tools that you can use to optimize its performance.
Learn what changes drive the most conversions
GWO intuitive reports allow even the mathematically- challenged to quickly and easily identify and implement the best combination. A large part of this article, however, is aimed at those who want to go beyond the, albeit powerful, analyses that are possible using GWO alone. I'll export some of its raw data and process this information offline as outlined in Figure 2 and detailed below.