Web Services Management: A Standards-Based Common Architecture

As enterprises rush to deploy Web Services-based solutions, problems in managing the services may become a challenge to their success. In a service-oriented world, service-level assurances are the need of the hour—implying services need to be “managed” well. It is important to understand what Web Services manageability means to a business, to an administrator, and to an end user. Equally important is to design an infrastructure that can work across existing platforms and can interoperate with existing standards.

In the first part of this two-article series, we will focus on the expectations from a Web Services management infrastructure. In the second part, we will propose and discuss a common architecture for the same.

Introduction

The growing prominence of Web Services¹ technologies is likely to lead to their adoption by businesses as a preferred application platform. However, to derive consistent business benefits from these technologies, Web Services applications and platforms, among other characteristics, need to

• Provide guaranteed quality of service.
• Be easily usable by service administrators in a production deployment.

The above requirements mandate that the Web Services products be “managed” well—in terms of monitoring, control, and co-ordination. Only in a well-managed Web Services ecosystem can QoS² (Quality of Service) parameters be measured, manipulated, and enforced.

Gartner has highlighted the need for Web Services management solutions: “As the hype about Web Services grows within the IT industry, real solutions for managing cooperating Web Services must emerge, or failures will ensue.”

Notably, there is an emergence of new application architectures based on Web Services technologies, as opposed to earlier monolithic or centralized architectures. Therefore, IT management technologies, too, have to evolve to keep up with the new requirements these architectures impose. Another diversity in the Web Services world is the parallel growth of J2EE (Java^TM 2 Enterprise Edition)-based and Microsoft^TM .NET-based deployments. A management strategy must encompass these two major platforms, as well as any upcoming platforms for Web Services. Also, to reduce the costs of its adoption, a management infrastructure must offer simplicity of use for administrators and, preferably, integration with existing management standards and products.

In the following discussion, we will analyze Web Services management requirements and, based on that, propose a platform-agnostic, common architecture for a Web Services management infrastructure compliant with suitable and popular standards.

We will focus on the expectations from a Web Services management infrastructure in this article and in the second part we will propose and discuss a common architecture for the same.

Gathering Requirements

A Web Services management infrastructure would be typically deployed and operated by the “Web Services administrators.” The administrators, normally part of an IT organization, would manage different sets of Web Services components depending upon the hosting configuration. Therefore, the requirements on what a Web Services management infrastructure must comprise will become clearer if we first take a look at various Web Services hosting paradigms and responsibilities of administrators in each of them.

Web Services hosting scenarios and administrator responsibilities

ASP³ (Application Service Provider)-hosted Web Services

In this scenario, an independent ASP is used by a Web Services provider to host Web Services of, for example, a small business. A Web Services provider in this case is a company that develops and deploys the Web Services for the business. The service administrator manages the Web Services development environment at its end as well as the deployment environment at the ASP end. The ASP thus has to provide a management access to a part of its environment to the administrator (See figure 1(a)).

Service Provider hosted Web Services

The Web Services provider itself hosts the Web Services here. This is a simplistic set up where the service administrator manages services deployed within the purview of the service provider’s environment (See figure 1(b)).

Mediated Web Services

This is another realistic configuration where Web Services are listed at a “service broker” intermediary but hosted at the service provider’s end. The service administrator in this case manages the service provider’s hosting environment as well as a part of the broker’s environment. An example of an intermediary is a portal that lists Web Services in a particular domain and lets users search and access them (See figure 1(c)).

Web Services client management

A variation of the preceding scenarios is the case where a portion of the Web Services client also needs to be managed remotely by the service administrator. (In this case, a Web Service may be running at the client end.) An example of a client that needs to be managed is the TV set-top box that can be manipulated remotely to let it decode only a limited number of channels (See Figure 1(d)).

Outsourced Web Services management

Another scenario, which takes the concept of services to the next level, is where some service providers offer Web Services management itself as a service and Web Services providers outsource management to these service providers. Thus, in this case, third-party providers manage a Web Services provider’s infrastructure and services.

Web Services Management: Common Administration Aspects

From the previous discussion, the following common points can be derived about general administrative responsibilities in the various Web Services hosting scenarios:

1. A Web Services administrator needs to manage a Web Services platform as well as the Web Services that are deployed on the platform.
2. The administrator also needs to manage the systems and the network in a limited way, considering that some of the service QoS parameters are dependent on them.
3. The management responsibilities may span remote sites—possibly across the Internet.



Figure 1: Web Services hosting scenarios

Functionality expected from the Web Services management infrastructure

Let us go into the details of what is expected from a Web Services management infrastructure—including what is not so obvious from the hosting scenarios. These requirements will lead us to a proposal on the Web Services management architecture.

1. As mentioned above, Web Services management can be done remotely—even across the Internet. This means that management information preferably must travel over a HTTP(S) protocol.
2. The management information should be able to be processed by another application using open APIs or data handlers. Data handlers, in this case, are typical loosely coupled XML document processors. This enables plugging in independent components to take appropriate action based on the state of the Web Services ecosystem.
3. The management technologies and products must be agnostic to the Web Services platforms: It is reasonable to expect that an administrator will manage Web Services deployed on platforms as varied as Microsoft^TM .NET or any J2EE-based Web Services platform, for example, BEA’s Weblogic^TM server—sometimes a mix of them at the same time. So the management infrastructure needs to be capable of handling all Web Services platforms as long as their management information has a compatible data format and this information is transported over a common protocol.
4. There are existing standards in the management domain. SNMP⁴ is a protocol widely used for managing network devices. Java^TM Management Extensions (JMX) from Sun Microsystems is an open technology for management of Java based enterprise software components. DMTF (Distributed Management Task Force) has defined the Common Information Model (CIM), which is a common data model of an implementation-neutral schema for describing overall management information in a network/enterprise environment. It would be desirable that a Web Services management infrastructure reuses these standards, so that information of other components in the ecosystem (that already use these standards for management information, for example, the OS and network devices) and information of the new Web Services components can be processed and presented in a common way to the administrators. Thus, the existing investment in IT management can be preserved. Also, the maturity of existing standards can be leveraged by adopting them.
5. Key functionalities: The basic functionalities of a Web Services management product would be:
• Platform monitoring: Monitoring and reporting the status of the Web Services platform—for example, whether the platform is running or stopped, whether more Web Services can be deployed on the platform, and so forth.
• Service monitoring and provisioning: Monitoring and reporting the status of the Web Services deployed as part of a business application—for example, whether the service is in running, paused, or in a stopped state.
• Life cycle and configuration management: Deployment, undeployment, configuration, and status change of Web Services on the platform—for example, changing the state of a Web Service from an “initial” state to a “running” state so that it can accept client requests.
• Performance management and service metering: Throughput, usage, and response time statistics reporting for the platform and for individual Web Services; Audit trailing for Web Services usage.
6. Additional desirable functionalities: Additionally, a Web Services management infrastructure may also provide:
• High availability: Fault detection in a Web Service during runtime and consequently switching to a failover deployment of the Web Service.
• Mediation: It may provide reliable messaging or even routing of a Web Services request to an appropriate service instance based on its usage characteristics.
• Billing and payment integration: Integration with billing and payment systems based on Web Services usage statistics.
• Subscription: Subscription-based access to Web Services.
• Security and transaction configuration: Integration with Web Services security and transaction infrastructure for configuring them. The security and transaction implementations may themselves be driven by separate specifications, but we need to have a common way of configuring these functionalities and management implementation is the obvious place to do it.
• Publishing: Automatic rule-based registration of Web Services to directories, as in a UDDI registry.
• Orchestration: Dynamic composition of Web Services to create Web Service workflows exposed as new or composite Web Services.
• Integrated systems management: Relevant Operating System and hardware performance statistics integrated in a common console for display.
• SLA and policy systems integration: Integration with SLA⁵ (Service Level Agreement) and policy management systems. In this case, management data is used to derive the actual values of SLA and policy parameters that can be used to compare it with agreed-upon SLAs and pre-defined policies. This enables monitoring and enforcement of SLAs and policies.
• Versioning: Versioning of server components that are part of the Web Services implementation.
• Notification: Notification of relevant management events via multiple channels.

Industry Standards

As identified in the previous section, standards conformance is a desirable aspect of a Web Services management infrastructure. At this point, let us concretely identify the data models and components that would comprise a typical web service management infrastructure and attempt to “standardize” them to arrive at a common architecture.

Data model

The Web Services management data needs to be modeled in a commonly accepted standard. New models can be defined or existing models can be reused. SNMP defines a data model (based on ASN.1⁶) for network devices. CIM from DMTF defines an object-oriented and extensible data model, which ostensibly is more suitable for generalized management purposes. For example, application management characteristics are defined in the CIM hierarchy. Web Services management characteristics can be derived from these application management characteristics and extended to include management data that make sense particularly for Web Services. An example of a Web Services specific management data is “service end point URL.” Thus, “Web Services” will become a child node to “application” in the CIM hierarchy. Similarly, the Web Services platform management can also be characterized using CIM, by extending the appropriate elements in the CIM hierarchy.

The management operations to be defined would include those related to deployment, usage characteristics, status reporting, and other advanced management functionality. These data models will ideally be standardized and accepted across vendors. Therefore, it is proposed here to accept CIM as the data model for the Web Services management infrastructure. Another factor that favors the adoption of CIM is that it is already a widely accepted standard across the industry for various management functions and is governed by a reputable independent body. For more information on CIM, refer to the DMTF Web site (www.dmtf.org).

Instrumentation

A Web Services platform would have to be instrumented to send and receive CIM management data at appropriate times. This process is proprietary to the platform and would be done by the platform vendors. However, the concept of interceptors can be used commonly across all platforms. Interceptors act on Web Services requests and respond to generate appropriate management information. They are generally provided by all platforms—each in their own way. Also, in the case of J2EE-based Web Services platforms, generic adapters can be built to simplify conversion of JMX data into CIM formats.

Web Services themselves must also be instrumented to generate or accept relevant management information. However, the platforms are expected to provide simplified interfaces for the service implementations for this purpose.

Transport protocol and data format

HTTP is the proposed transport protocol for the management data so that it is accessible over the Web. XML, due to its wide acceptance, is the proposed data format. DMTF has already defined XML/HTTP (both 1.0 and 1.1) bindings for the CIM data model—in the form of the WBEM (Web-Based Enterprise Management) standard. Thus, WBEM includes:

• A data model, the Common Information Model (CIM) standard
• An encoding specification, xmlCIM Encoding Specification
• A transport mechanism, CIM Operations over HTTP

WBEM can be extended in a similar fashion as CIM is extended. Hence, WBEM is proposed to be accepted as the standard for Web Services management infrastructure.

More on WBEM: The CIM specification is the language and methodology for describing management data. The extensible CIM schema includes models for Systems, Applications, Networks (LANs), and Devices. The CIM schema will enable applications from different developers on different platforms to describe management data in a standard format so that it can be shared among a variety of management applications. The xmlCIM Encoding Specification defines XML elements, written in Document Type Definition (DTD), which can be used to represent CIM classes and instances. The “CIM Operations over HTTP” specification defines a mapping of CIM operations onto a HTTP protocol that allows implementations of CIM to interoperate in an open, standardized manner and completes the technologies that support WBEM. For more information on WBEM, refer to the DMTF Web site (www.dmtf.org).

Presentation

The presentation of the management information should be possible in GUI consoles, Web browsers, and/or in PDA consoles. XSL transformation stylesheets can be developed for the CIM/XML data and can be used for dynamically creating HTML or other data for rendering the consoles, browsers, or PDAs. All supported statistics and controls thus can be viewed and operated upon. Another possibility for the presentation layer is to integrate with existing popular management platforms. For example, Web Services management consoles can be integrated with HP Openview^TM user interfaces. Separation of data from presentation gives the flexibility of having disparate suitable consoles processing the same management data format.

Summary

In this first part, we discussed the expectations from a Web Services management platform and the standards relevant to Web Services management. Based on this analysis, in the second part, we will propose a common architecture for Web Services management.

About the Authors

Pankaj Kothari is a technical lead at Hewlett-Packard, India. He has been involved in designing and developing enterprise middleware and Web Services products such as e-speak and HP Application Server. He has also contributed to the architecture and implementation of solutions based on these technologies in various domains for more than five years. He can be reached at pankaj_kothari@hp.com.

Ravi Trivedi holds a Masters degree in Computer Science from the Indian Institute of Science, Bangalore. He is a technical lead at Hewlett-Packard, Bangalore. He is an expert group member for HP in JAXR (JSR 93) and a committer for open source UDDI4j (www.uddi4j.org). He is involved in developing core Web Services infrastructure and solutions based on these. He can be reached at ravi_trivedi@yahoo.com.

Acknowledgements

The authors would like to acknowledge the contributions of Srinivas Varadarajan, Ashish Chitkara, Manoj Seth, Shyam Bijadi, and Jainendra Kumar at Hewlett-Packard, India for their insightful reviews.

¹ Web Services is a term used to describe components and services that are addressable and available using Internet technologies.

²QoS (Quality of Service) specifies a guaranteed response time and throughput level for Web Services users.

³An application service provider (ASP) is a company that offers enterprise access to a Web Services hosting infrastructure that would otherwise have to be located in their own enterprise computers.

⁴Simple Network Management Protocol (SNMP) is the protocol governing network management and the monitoring of network devices and their functions.

⁵A Service Level Agreement (SLA) is a contract between a Web Services provider and a customer that specifies, usually in measurable terms, what Quality of Service the Web Services provider will furnish.

⁶ASN.1 (Abstract Syntax Notation One) is a standard way to describe a message (a unit of application data) that can be sent or received in a network.

Disclaimer : The views expressed here are those of the authors and does not necessarily represent that of Hewlett Packard.