Relationship Between Event and Control Streams
First, look at contracts between a single subscriber (client) and a single publisher (server). The subscriber sends a message containing a subscription to the publisher; the subscription specifies the conditions under which the publisher will send a message to the subscriber. The subscriber may send subsequent messages updating the subscription.
Click here for a larger image.
The stream of messages from the subscriber to the publisher is a control stream. The stream tells the publisher what to look out for. Subscriptions are specified in terms of a formula or model or in terms of examples for use in machine learning. Subscriptions may include service-level agreements (SLAs) that specify, for example, the maximum delay between the occurrence of a situation and the delivery of a message to the subscriber identifying the situation. Clients find services for EDA just as they do for SOA: through brokers.
Click here for a larger image.
Models and machine learning techniques can be sophisticated.
Click here for a larger image.
Simplistic subscriptions result in publishers generating irrelevant messages. Subscribers then must devote computational cycles to extract the information they need from event streams. Model-based subscriptions allow publishers to carry out more computation—executing models—to ensure that only appropriate messages are generated. This reduces the volume of messages from publisher to subscriber and reduces the computational load on subscribers. Thus, the use of sophisticated models in subscriptions allows for the balancing of computational load across multiple processes and a reduction in communication volume.
In many cases, a subscription is of the form: "Tell me when something unusual happens." The subscription may specify a model of normal behavior. Any violation of the model is deemed anomalous and results in an event message being sent. Alternatively, the subscriber may specify a formula that directly identifies anomalous behavior.
Click here for a larger image.
The two kinds of subscriptions are not fundamentally different. In some cases, specifying what is normal is easier than specifying exceptional situations; in other cases, specifying anomalous situations is more convenient.
The next section presents some examples of the first type of specification: a model specifies normality. Later, you'll see examples of the second type, in which anomalous behavior is specified directly.
A Model Specifies Normality
Consider a security application in which an organization wants to be alerted if its infrastructure is being attacked. Specifying normal behavior (by providing a large number of examples of normal situations, for instance) is often easier than specifying situations that indicate attacks. A violation of normal behavior may not be the consequence of an attack but nevertheless is a situation that needs to be investigated.
In a commodity-trading application, a model may specify relationships between prices of a commodity at different places, prices of substitutable commodities, costs of shipment between different places, exchange rates, and so forth. A variation from the model may signify a threat or opportunity that is worthy of exploration.
Next, consider examples in which anomalous behavior is specified directly. The previous article presented a simple example of a publisher in the form of a temperature sensor. The subscription includes a forecast—a model—of temperature over the next 24 hours. The sensor sends a message containing the measured temperature when the measured temperature deviates from the forecast temperature by more than a threshold value.
Click here for a larger image.
A subscription in a stock-trading example contains a model of value-at-risk of the portfolio and a threshold for this value. The publisher sends an event to the subscriber when this threshold is exceeded. A subscription in an accounting application contains a model—a plan—of expenditures and revenues. The publisher sends an event when actual expenditures or revenues deviate from the plan.
Click here for a larger image.
When reality deviates from a model, a subscriber either updates the model by sending control messages to the publisher or simply receives messages notifying it of prescribed types of deviation. In the former kind of subscription, human intelligence (usually) is required to investigate the deviation and determine whether it is significant. An example of the latter kind of subscription would be one message received when revenues are below plan and another message received when expenditures exceed the plan—and the subscriber may invoke a computational process to respond to the message. In all these applications, a subscriber can conclude that the subscriber's model fits reality until the subscriber receives a message indicating the contrary.
EDA Model in Use
Consider a couple of simple examples that illustrate the ubiquity of the EDA model. Do you know of any organization in which a vice president (VP) tells the CEO about an unusual situation—say a manufacturing plant burns to the ground—only if the CEO asks?
Click here for a larger image.
Effective organizations use (usually implicit) contracts that specify what should be communicated and when. The VP is responsible for telling the CEO that a plant has burned; the CEO is not responsible for extracting that information from the VP. Pharmaceutical companies are responsible for telling the FDA when drug trials result in deaths; the FDA is not required to poll the pharmaceuticals to obtain this information. The shared model—the shared expectation—about what is to be communicated is an implicit subscription.
Click here for a larger image.
Communication within any organization—say the healthcare system—is based on shared expectations between people and institutions. These shared models, or shared expectations, are critical to the success of the organization. What the pharmaceuticals tell the FDA and when, what the FDA tells doctors and when, what doctors tell patients and when, all depend on shared expectations.
Click here for a larger image.
IE 7
Firefox 2.0
