Statistical and Financial Considerations in Website Optimization
The ROI / Business case for testing
Sometimes it's hard to convince your boss that your company should invest precious resources into running content experiments. One way to communicate the business value of testing is to run the numbers.
As an example, let's say your company currently acquires customers or sells products online. Let's assume every year, 1 million visitors reach your site and 5% proceed past your home page, 10% of those proceed past your product/service details page, and 20% of those ultimately become customers. Let's also assume the average sale is $250. This would equate to 1,000 customers and $250,000 in revenue for every 1,000,000 visitors. Not too shabby right?
So at the end of the day, using experiments to find better-performing content can dramatically increase your sales without increasing your spending. After using Website Optimizer to improve your ROI, you might even find your boss asking you why you aren't testing more often.
User Experience Testing
Because usability is a huge factor in conversion rate improvement, user experience (UX) people are usually heavily involved in the development of a written test plan (helping to determine the elements to be tested, along with the specific alternatives for each element to be considered). Remember that successful testing all of the elements on a page may require the collection of a great deal of raw data. So, if UX testers can correctly eliminate the need to include some elements from your initial optimization testing, their efforts will shorten the time required to collect a statistically significant amount of data.
UX experts can also readily construct the awareness, interest, desire, action (AIDA) decision process steps for your business.
- AIDA is a model of consumer behavior that traces the sequence of cognitive events leading to a purchase decision or other action; also called hierarchy of readiness. For example, in a political campaign, one first becomes aware of the candidates. After receiving additional information, an interest develops in one or more candidates, eventually resulting in a desire to see one candidate elected, and the act of casting a vote for that candidate. The AIDA model is used by marketers as a guideline for creating communications. This requires an understanding of where the market for a product currently lies among the AIDA continuum. Marketing of an innovation requires building awareness. Marketing of an established product may require building desire. Recipes that present new ways to use established brands are one way often used to build desire for an existing product.
UX practitioners are generalists. They may have been involved in the design of many websites on a variety of topics. For this reason, it is important to team them with a subject matter expert (SME). Without the support of someone knowledgeable in your industry or business, UX practitioners may miss important aspects of your conversion process or business goals.
UX people are usually good at the functional and architectural aspects of your designs (i.e., common usability issues that are likely to affect all of your visitors). They are generally weaker on the content issues, such as text, copy, marketing message, and graphical design.
The question is should you deploy a Website decided upon by a conference room of experts (e.g., usability testers, marketing gurus, etc.), by a self contained tool like Website Optimizer, by an advanced statistical analysis like the one outlined above or some combination of them. The answer, as you may have guessed, is it depends. It depends on your budget (dollars and time), the level of experience and expertise in such matters of your people (IT, Finance, Marketing, etc.), and the importance of the ROI you think you can achieve.
Remember, without a positive ROI, advertising is a cost, not an investment!
- Tullis et al Measuring The User Experience, Morgan Kaufmann (2008)
- Ash, T. Landing Page Optimization, Sybex (2008)
- Eisenberg et al The Complete Guide to Google Website Optimizer, Sybex (2008)
- Clifton, B. Advanced Web Metrics with Google Analytic, Sybex (2008)
- Albright, S. C. Learning Statistics with StatTools, Palisade (2003)
- King, A. Website Optimization, O'Reilly (2008)
- Carlberg, C. Business Analysis with Microsoft Excel 3rd Ed, Que (2007)
- Corning, P. Nature's Magic, Synergy and Fate, Cambridge University Press (2003)
- Motulsky, H. Intuitive Biostatistics, Oxford University Press (1995)
- Website Optimizer Overview
- Choosing a statistical test:
- Content testing ideas:
About the AuthorMarcia Gulesian is an IT strategist, hands-on practitioner, and advocate for business-driven architectures. She has served as software developer, project manager, CTO, and CIO. Marcia is author of well more than 100 feature articles on IT, its economics, and its management. Scroll to the bottom of her blog for links to many of these articles.
Page 5 of 5