Measuring the Benefits of Ajax
How Is Ajax Different?
Where Do the Benefits of Ajax Come From?
Often, in business, decision makers are interested mainly in how information technology can reduce costs, or make better use of information assets. The benefits of Ajax seem to come more out of the cost-containment arena than the latter. The question becomes "Where do these cost savings come from and how can we quantify them?"
1. Potentially Measurable Benefits
These are benefits that can be measured and expressed in terms of dollars and cents without much difficulty. Regardless of the quality of your Ajax UI, you will look to these metrics to estimate value. They include:
- Time spent waiting for data to be transmitted: Time is money. Over many repetitions, the time employees spend waiting for the page to load can add up to significant costs.
- Time spent completing a particular task: Increased efficiency in the user interface can often mean that time is saved at the task level, offering opportunities for concrete cost savings there.
- Bandwidth consumed for the entire task: The cost of bandwidth does not increase linearly, but does increase as the company invests in larger-capacity Internet connections and new hardware to accommodate greater server loads. A firm's cost structure for bandwidth depends on the scale of their operation and these capital investment needs. That being said, the cost of bandwidth can be measured if this cost structure is known. If repetitious tasks consume a lot of bandwidth, these costs can escalate dramatically. The amount of bandwidth consumed also has implications for time savings.
2. Hard to Quantity Benefits
Some of the benefits associated with good user interfaces are qualitative and difficult to measure precisely. This isn't to imply they are not of financial value, but many business benefits are hard to quantify, and seasoned IT managers will know intuitively they can translate into significant bottom line savings. For Ajax, the opportunities for streamlining the interface are limited only by our imaginations, and it should also be noted that it's still possible to design a terrible UI with Ajax that doesn't benefit from any of the following:
- Steps to complete a task: Reducing the number of steps has implications for the amount of time consumed but also for the number of opportunities for error. Fewer errors mean cost savings down the road when these errors would have to be manually corrected.
- Familiar user interface: Quite often these days, Web-based applications are used to replace desktop applications that had superior user interfaces. The benefits of offering users a similar or even just a familiar user interface to what they use on the desktop means lower training costs, fewer errors, and greater initial productivity.
- Improved application responsiveness: More responsive applications can improve productivity not just by reducing "wait," but by promoting a more fluid, uninterrupted workflow. In a responsive application, users can move rapidly from one action to another as quickly as they can visualize the workflow. Less responsive applications can defeat the user's workflow visualization by forcing them to continually wait for program information.
The Ajax Test Case
To explore the potential for cost savings, look at a sample application that was built to study the differences between the traditional Web architectures and Ajax. I call this application the "United Chemicals Sales Tool (UCST)." United Chemicals is a fictional company that sells household chemicals. The UCST allows sales personnel to transact sales and maintain a general database of customers. The application is written in ASP, and is connected to a database of roughly 10,000 customer records, 60,000 sales records, and a catalogue of about 600 products. This is meant to represent a controlled version of a scenario you might see in the real world.
Two versions of the UCST application were written. One version was developed using a traditional architecture. The user interface resembled one you might see if you had used the Microsoft Data grid to display and edit records. The other version used an Ajax component, the EBA Grid Control developed by eBusiness Applications (http://developer.ebusiness-apps.com) to edit data in an Excel-like interface. In almost every other respect, these applications function identically. For this to be a fair comparison, several guidelines were used in the execution of the test case:
- Five test users were required to enter exactly the same amount of data in each application for each task.
- The traditional architecture version would only post back to the server when absolutely necessary.
- Microsoft Fiddler (http://www.fiddlertool.com) and a stopwatch were used to gather relevant metrics.
- Test users had at least a moderate level of computer knowledge.
- Test users received the same training in both versions of the application beforehand.
The United Chemicals Sales ToolAjax Version
Two tasks were designed to be representative of a sales entry tool. Task 1 was to add a sale to an existing customer. Users had to locate the customer from the list, and process a sale for that person. Task 2 was to add a sale to a new customer. This task involved more steps and required the user to enter the data, perform a search, and then enter the sale information. Both tasks together had 10 steps. Users performed both tasks on both versions of the UCST application.
The United Chemicals Sales Tool test application can be found here: http://www.ajaxinfo.com/ucst/index.html
Traditional vs. AjaxThe Results
The metrics taken for this study were bytes transferred, total time consumed by the task, and Microsoft Fiddler's estimation of the time it would take to transfer the same data in the same number of HTTP requests to a location on the US West Coast from our position.
Total (Task 1 and Task 210 Steps)
|Traditional (Average)||Ajax (Average)||Performance Increase||Performance Increase (%)|
|Estimated Transmission time to US West Coast (56k) (seconds)||293.45||94.44||199.01||68%|
As expected, in every test there were significant performance improvements across the board in the Ajax version. The greatest improvements were in bandwidth and network traffic efficiency gains. This resulted in improved responsiveness, allowing the user to move more quickly through the application. Overall, there was a 73% improvement in the number of bytes transferred in the Ajax application over the traditional version. This was due to the fact that that server requests were made only for the data that was needed, not for the entire page. Microsoft Fiddler also predicted that there would be a 68% overall time improvement in transferring that information to a location on the US West Coast given the number of HTTP requests, bytes transferred, the latency of that connection, and the bandwidth afforded by a simple 56k modem.
More importantly than bandwidth considerations was the direct benefit to users who saved on average 32% of the time required to complete the tasks in the Ajax version. Had those users been connecting to a remote location instead of a local server, Fiddler predicted the time savings could have been much more dramatic (around 68%). We can take these numbers now and extrapolate to see how large enterprises could be affected when these actions are repeated over and over.
The Benefit to Business
Using the test application with this 10-step process, one already can see that the potential for time and bandwidth savings is dramatic. It's possible also to express these savings in dollars and cents, given certain assumptions and using basic math. You calculate these savings based on the labor rate, the number of transactions, and the time savings per transaction.
Ajax Cost Savings = Hourly Labor Rate X (Seconds Saved per Transaction X Number of Transactions per year) / 3600
Looking at a conservative potential time savings of 36 seconds per transaction, if a business performs 50,000 of these transactions per year, and a labor cost of $20/hour, we see a total labor savings of $10,000 per year, or 500 man hours, on this transaction type alone.
Given the Microsoft Fiddler predicted time savings of 199.01 seconds per transaction for a remotely hosted application, you see a more aggressive cost reduction of approximately $55,281 per year for this transaction type, or 2,764 man hours.