Put Some "BAM" in Your Next BizTalk 2004 Project

BizTalk Orchestrations compose the physical implementation of a Business Process. In a typical Orchestration, you may share data with an external system or invoke a Web service. Although your clients may be pleased with the new automation, automating the process with an Orchestration oftentimes is only part of the solution. Not long after an Orchestration is implemented, clients begin asking questions like "how long does it take to move from step 1 to step 5?" or "what is currently in process?"

To gather such metrics on the steps within your BizTalk 2004 Orchestration, check out Business Activity Monitoring (BAM). With BAM, you can log numeric, textual, and chronological data about any step in an Orchestration to a SQL Server database and even an Analysis Services Cube. With the logged data, you can build reports or custom monitoring applications for BizTalk Orchestrations.

Using a sample application, this article shows you how to set up BAM for your own BizTalk Orchestration. Before diving into the sample application, however, here's an overview of BAM.

What Is BAM?

BAM is composed of the following items:

1. The BAM Definition Workbook is an Excel add-in built into an Excel document, which you use to build BAM definition Activities and Views (more on these in a moment). You use the Excel spreadsheet to create a BAM Definition.
2. The BAM API is used by BizTalk and your BizTalk Orchestration. The API is divided into a synchronous (DirectEvent Stream) and an asynchronous (BufferedEvent Stream) variety. (The article discusses the BAM API in more detail later.)
3. A set of SQL Server databases store BAM data in different stages. Figure 1 shows the BAM Primary Import database, a Staging database, and an Analysis Cube.

Figure 1. Simplified View of All BAM Server-side Components

As you can see, BAM is separate from the BizTalk Service. BizTalk interacts with BAM through the BAM API. The BAM API updates the BAMPrimartImport database. Separately, a set of DTS packages are responsible for archiving the data, staging the data in BAM Star Schema, and interacting with Analysis Services.

You don't have to wire BAM into your Orchestration to use BAM features. However, if you don't like how BAM logs particular pieces of information, you can build custom features at different levels within BAM, including making BAM part of the Orchestration. (The default and custom behaviors of BAM are discussed later.)

You begin BAM development by creating a BAM Definition using the BAM Excel document. As stated earlier, you set up Activities and Views in the BAM Definition. Activities contain Items, and Views contain Measures. Items are the things you log, and Measures define how you slice and dice the Items.

Before learning the BAM Definition, look at the Orchestration sample application.

The Sample Application

Figure 2 shows the sample Orchestration.

Click here for a larger image.

Figure 2. Orchestration of Sample Application

Two ports are set up using the File Adapter. A Receive File adapter Port initiates the Orchestration. The receive message contains three pieces of information:

• IDValue: a representation of a unique value identifying the Orchestration
• NumIterations: a parameter passed to a class in an assembly
• Descrip: a description of the message

A Construct Message utilizes a transformation to create the message for the Send Port. Next, the Orchestration invokes a class from a .NET assembly, invokes some BAM API code, and then posts a message to the Send port.

Everything is now in place to begin BAM development. The first step is creating a BAM Activity.

Creating a BAM Activity

The BAM Definition Excel spreadsheet is engineered for use by information workers rather than developers. So, defining Activities requires no coding. Start by making a copy of the BAM.xls spreadsheet located on the following path:

C:\Program Files\Microsoft BizTalk Server 2004\Tracking

The BAM Add-in appears as a menu option when you open the spreadsheet. See Figure 3.

Figure 3. Sample Menu

When you select BAM Activity, the BAM Activity wizard appears. From the Activity wizard screen, select "New Activity". The dialog in Figure 4 appears.

Figure 4. Dialog from New Activity Selection

As you'll see later, the Activity Name will appear in SQL tables when the BAM definition is loaded. Items for an Activity fall into the categories shown in the dialog in Figure 5.

Figure 5. Dialog of the Item Categories

All but Business Milestones are self-explanatory. Business Milestones are datetime value fields. Use Milestones when you want to record the time a step in your orchestration executed.

As you define the items in your Activity, keep in mind that each instance of your Orchestration creates one record in the SQL database. So, for example, if you have a loop in your Orchestration recording a Milestone, the most recent datetime value will be recorded in the database.

The sample contains two Activities: IterationTracking and IterationTrackingDirect. Once you've created your Activities, you can create Views to aggregate the Activity Data collected by BAM.

Creating a BAM View

The BAM View Wizard also can be accessed from the BAM Excel menu. Once you access the BAM View Wizard and elect to create a new View, the dialog in Figure 6 appears.

Figure 6. New View Dialog

As you can see, you can select multiple Activities. Once you've created your View, you must define Dimensions and Measures. Think of Dimensions as the "what" and Measure as the "Value." Measures allow you to aggregate Items in your view; thus, measures are confined to numeric Items you collect in your BAM Activity (see Figure 7).

Figure 7. Measures Dialog

Dimensions explain what a Measure represents. Dimensions can be composed of combinations of Activity Items. The Dimension dialog for a Business Milestone appears in Figure 8.

Figure 8. Dimension Dialog for a Business Milestone

You are now ready to deploy the BAM Definition.

Deploying the BAM Definition

You can deploy your BAM Definition by using a command-line utility called BM.exe. BM.exe accepts the definition in Excel or an exported XML format. I recommend exporting to XML, mainly because if the computer you are running BM.exe from does not have Excel loaded, you will get an error message. Figure 9 shows the output of the BM.exe utility.

Click here for a larger image.

Figure 9. Output of the BM.exe Utility

When BM.exe deploys the BAM Definition, it creates SQL Tables and, if you've defined Views, Analysis Services Cubes. BM also creates Data Transformation Services (DTS) packages that you can modify to move the BAM data into the desired place.

As BAM writes the Items defined in your Activities, it places the data in a number of databases, beginning with the BAMPrimaryImport database.

BAM Databases and DTS Packages

Figure 10 displays the tables created in the BAMPrimaryImport database after the BM.EXE utility executes.

Click here for a larger image.

Figure 10. Tables Created in the BAMPrimaryImport Database

BAM will copy the data to a particular table depending on whether an Activity is "Active" or "Completed." Tables are created with the appropriate keys and unique indexes depending on how you call the API or define the Tracking Profile (more on the BAM API and Tracking Profiles later). Figure 11 shows some data from one of the tables in the sample application.

Click here for a larger image.

Figure 11. Data from One of the Tables

In addition to tables, BAM also creates a set of stored procedures for use by the BAM API (see Figure 12).

Click here for a larger image.

Figure 12. BAM Stored Procedures

Figure 13 lists the DTS packages created when the BAM Definition is deployed.

Click here for a larger image.

Figure 13. DTS Packages

The DTS Packages perform actions like archiving and populating Analysis Services Cubes. You can modify the packages to perform other operations.

Once you've created and deployed your BAM Definition, you have two implementation options:

• Use the BAM API to collect your BAM Activity Items.
• Collect your BAM Activity Items using a Tracking Profile.

Calling the BAM API

As stated earlier, the BAM API has a synchronous (DirectEventStream) and an asynchronous (BufferedEventStream) form. The following code from an Expression Icon in the Sample Orchestration illustrates how you invoke the BAM API:

bamDirectEvent =
   new Microsoft.BizTalk.Bam.EventObservation.
   DirectEventStream("Integrated Security=SSPI;Data Source=.;
                      Initial Catalog=BAMPrimaryImport",0);
bamDirectEvent.BeginActivity("IterationTrackingDirect",IDValue);
bamDirectEvent.UpdateActivity("IterationTrackingDirect",
   IDValue,"IDValue",IDValue,"IterationNumber",NumIterations,
   "Descrip",Descrip,"RecordDate",System.DateTime.Now);
bamDirectEvent.EndActivity("IterationTrackingDirect",IDValue);

The DirectEventStream and BufferedEventStream have the same set of functions. BeginActivity initiates the logging, UpdateActivity continues the logging, and EndActivity ends the logging.

The main differences between DirectEventStream and BufferedEventStream are:

• The BufferedEventStream uses the BizTalk Messagebox and then writes the logging data to the SQL database.
• DirectEventStream writes directly to the SQL database.

The BAM API classes are located in the "C:\Program Files\Microsoft BizTalk Server 2004\Tracking" folder, in an Assembly called Microsoft.BizTalk.BAM.EventObservation.

The BAM API also includes functionality for redirecting BAM Activity to other data sources in a set of BAM Interceptor functions and Interfaces.

As stated earlier, the BAM API was one option for implementing BAM. The easiest way to implement BAM in your orchestration is to create a Tracking Profile.

Setting Up a Tracking Profile

The Tracking Profile Editor is one of the BizTalk administration tools. Select File -> New, and then select your Orchestration to create a new Tracking Profile. A mock-up of your Orchestration will appear in the tool (see Figure 14).

Click here for a larger image.

Figure 14. Tracking Profile Editor

To link your Orchestration to an Activity Definition, right-click on the Orchestration and select Import Activity Definition. When you import the Activity Definition, a screen like the one in Figure 15 will appear.

Click here for a larger image.

Figure 15. Import the Activity Definition

Map the Activity Items to steps in your orchestration in the following manner:

1. Milestone Items can be mapped by dragging and dropping the Orchestration Icon you want to relate to the milestone.
2. To map other Items, you must connect the Item to data in a message. To do this, right-click on Icons (which create messages), select the appropriate schema, and drag the appropriate element in the message to the Activity Item.

To deploy the Profile, select the Deploy option on the File menu.

Tracking Profiles still utilize the BAM API. When you use a Tracking Profile, instead of writing code BizTalk performs the API calls for you.

Now that you have completed your BAM implementation, consider some final issues.

Using What You've Logged

You won't realize the true value of BAM until you use the BAM logging data with tools like Reporting Services and Notification Services.

If you've included things such as Purchase Orders and other keys in your BAM data, you can combine the BAM data with data in other databases. For example, you can modify the DTS packages to divert your BAM data to another database that includes more detailed information about your business process, and then generate reports using Reporting Services.

If you want to generate special notifications based on your data, you can build the Notification using Notification Services.

Any other tool capable of reading SQL Server data also can utilize the BAM logged data. If you want to experiment, start with the BAM Definition spreadsheet. The spreadsheet has built-in functionality for reading the Analysis Services Cube data. You can use the spreadsheet to prototype your ideas.

Record Your BizTalk Orchestrations' Progress

BAM allows a developer to record the progress of your BizTalk Orchestrations for consumption by tools such as Reporting Services. BAM is straightforward to set up, and it can execute in a synchronous and asynchronous fashion depending on your performance requirements. Finally, BAM can be customized to write data to other databases to satisfy any reporting requirements.

Sources

• BizTalk 2004 Unleashed, Woodgate 2004
• "BizTalk Server 2004: A Review of the Business Activity Monitoring (BAM) Features"

Download the Code

To download the accompanying source code for this article, click here.

About the Author

Jeffrey Juday is a software developer with Crowe Chizek in South Bend, Indiana. He has been developing software with Microsoft tools for more than 12 years in a variety of industries. Jeff currently builds solutions using BizTalk 2004, ASP.NET, SharePoint, and SQL Server 2000. You can reach Jeff at jjuday@crowechizek.com.