BizTalk 2004 Pipelines allow you to customize the processing of XML documents received or sent via the various BizTalk 2004 adapters. Custom Pipeline components extend the behavior of Pipelines to include processing data of virtually any format. They can be a powerful solution if you support legacy systems that require integration with other products, but your legacy data format does not follow standards. It may contain carriage returns in odd places or records spanning any number of lines of text, for example. The only stipulation is that the data emerging from the Custom Pipeline must be some form of XML document.
At my company, Crowe Chizek, the time and billing system generates invoices as ASCII text files, which follow none of the standard formatting rules, such as tab delimited, fixed field length, comma delimited, etc. These files include formatting characters in the text. A separate program has to interpret the ASCII text’s formatting characters and print the invoice. Crowe Chizek’s time and billing users were spending a great deal of time printing invoices and routing hard copies with corrections to each other.
We decided to build an application to facilitate invoice creation from the time and billing system. Rather than reconstructing the business logic, which created the ASCII file, we decided to load the billing information directly from the ASCII file into our invoice facilitation application. We employed the capabilities of BizTalk 2004 to handle the ASCII files and perform workflow actions like posting email once a file has been processed. The BizTalk 2004 File Adapter worked fine for receiving the files, but we needed a custom Pipeline component and a Pipeline to process the incoming files. This article demonstrates the custom Pipeline component solution we employed.
Building the Custom Pipeline Component
As stated previously, pipelines are implemented in a BizTalk 2004 project for a variety of reasons, including encryption/decryption, splitting single XML documents into multiple documents, and converting flat files into XML documents. You begin building a Pipeline by adding a one to your BizTalk 2004 project (see Figure 1).
Figure 1. Add a Pipeline to Your BizTalk 2004 Project
To complete a Pipeline, you must select one or more Pipeline components from within Visual Studio .NET and add it/them to the Pipeline. None of the existing BizTalk 2004 Pipeline components satisfied our requirements, so we built our own.
A custom Pipeline component is a .NET assembly that implements the Microsoft.BizTalk.Component.Interop.IComponent interface, the System.Runtime.InteropServices.Guid property, and the ComponentCategory(CategoryTypes.CATID_Validate) property housed in the Microsoft.BizTalk.Pipeline.dll assembly. Samples in the BizTalk 2004 SDK and the BizTalk 2004 documentation illustrate how to implement the appropriate properties. This tutorial focuses on the areas specific to Crowe Chizek’s implementation and summarizes the items already documented in the BizTalk 2004 documentation.
IComponent contains a method called Execute. The Execute method is where the magic happens in the custom Pipeline component. In the Execute method of the Pipeline Assembly, an XML document replaces the data from the ASCII file. The following code enables you to gain access to the data from the ASCII file passed to the Pipeline:
IBaseMessagePart bodyPart = inmsg.BodyPart;
if (bodyPart!=null)
{
Inmsg is the IBaseMessage interface passed to the Execute function. IBaseMessagePart includes a function for accessing the data in the message pipeline. The GetOriginalDateStream function returns a Stream object. Stream objects are part of the .NET Framework. You can read the data from a stream by using methods like Seek and Read. The following example declares originalStrm as a Stream object:
originalStrm = bodyPart.GetOriginalDataStream();
Copying the ASCII data out of the Stream you’ve accessed via the IBaseMessagePart requires you to move the data into and out of byte arrays. The following code copies all of the data from the ASCII text file to a string variable called myData:
buffer = new byte[originalStrm.Length]; originalStrm.Read(buffer,0,Convert.ToInt32(originalStrm.Length)); myData = System.Text.ASCIIEncoding.ASCII.GetString(buffer);
To write the XML document back to the Pipeline, simply replace the Stream object with a new Stream object. The following sample code fills a MemoryStream object with the XML data and assigns the bodyPart.Data property to the MemoryStream object you loaded with the XML document:
//Change the data bodyPart.Data = memStream;
The only remaining task to complete the Execute method is to promote the message so BizTalk can properly identify it. The following code ensures that BizTalk can identify the Namespace for the message once the XML document passes out of the Pipeline:
// You must manually set the message type
systemPropertiesNamespace = @"http://schemas.microsoft.com/BizTalk/
2003/system-properties";
inmsg.Context.Promote("MessageType",
systemPropertiesNamespace,
_XMLMessageType);
If you did not promote the message before sending it out of the Pipeline, BizTalk could not route the message to the Receive Shape in your Orchestration.
Time to Debug
With the custom Pipeline component complete, we turn our attention to debugging. Our implementation was more complicated than the included sample code. To debug our solution, we utilized some of the BizTalk Pipeline debugging tools.
Setting up your project to debug a custom Pipeline component is not straightforward. BizTalk includes a command line utility for debugging Pipelines aptly named Pipeline.exe. The file is located in the ” Installdir:Program FilesMicrosoft BizTalk Server 2004SDKUtilitiesPipelineTools ” directory. To utilize it, you must set the options shown in Figure 2 under the debugging section on the Properties dialog of the Project.
Figure 2. Settings for Utilizing Pipeline.exe Debugging Utility
Command Line Arguments include the name of the Pipeline that uses your component, the name of the sample file to pass to your Pipeline, the ” -c” to display the resulting output document, and ” -v” command line switches to display additional data, which will be useful for troubleshooting.
We also implemented a destructor for the custom Pipeline component so we could freeze the results on the Pipeline.exe utility as we ran the debugger from Visual Studio. Because the Pipeline.exe utility doesn’t know the results of the Execute statement until you exit the Execute statement, you can freeze the execution of the utility by putting a breakpoint in the Destructor code of you custom Pipeline component object.
The IBaseMessage interface includes other things helpful for debugging and troubleshooting, including a property called Context. Context contains metadata about the message being sent through BizTalk. Because we used the File Adapter, the Context property includes information about the file collected by the File Adapter. The following code illustrates how you access the name of the file sent to the Pipeline:
originalFileNameObj = inmsg.Context.Read("ReceivedFileName",
@"http://schemas.microsoft.com/BizTalk/2003/file-properties");
if (originalFileNameObj == null)
If you debug using the Pipeline.exe utility, the file Context information is blank. The information is populated during runtime. The sample code captures only the name of the file, but you can capture a variety of other pieces of information from the Context. For example, you can view a list of message namespaces and Context properties by accessing the Configuration menu-Message Properties in the Health and Activity Tracking (HAT) application (see Figure 3).
Figure 3. Access Configuration menu-Message Properties in the HAT Application
Add a Custom Component to the Pipeline
This article began by creating a Pipeline in a Visual Studio.NET BizTalk project. The final step is adding the custom Pipeline component to the Pipeline in the BizTalk .NET Assembly. After creating any custom Pipeline component, you must deploy the pipeline to the C:Program FilesMicrosoft BizTalk Server 2004Pipeline Components directory so the custom Pipeline component can be added to the Pipeline in your BizTalk 2004 project. After you deploy the custom Pipeline component, you must add the component to the Pipeline component toolbar. To add the component, right-click the top of the Pipeline Component toolbar and select “add and remove items”, as shown in Figure 4.
Figure 4. Right-Click the Top of the Pipeline Component Toolbar
Select the Pipeline Component on the Pipeline Toolbar tab (see Figure 5).
Figure 5. Select the Pipeline Component on the Pipeline Toolbar Tab
You can now drag and drop the custom Pipeline component onto the Custom Pipeline (see Figure 6).
Figure 6. Drag and Drop the Component onto the Custom Pipeline
You can now compile and deploy the new pipeline.
Handle Non-XML Data in BizTalk Messages
Pipelines allow you a great deal of control over the processing of BizTalk 2004 messages. Custom Pipeline components allow you to handle non-XML data in BizTalk messages coming through an adapter. For turning non-delimited ASCII text files into a BizTalk-consumable XML format, a custom Pipeline component is your best option.
Download the Code
To download the accompanying source code for this article, click here.
About the Author
Jeffrey Juday is a software developer with Crowe Chizek in South Bend, Indiana. He has been developing software with Microsoft tools for more than 12 years in a variety of industries. Jeff currently builds solutions using BizTalk 2004, ASP.NET, Sharepoint, and SQL Server 2000. You can reach Jeff at jjuday@crowechizek.com.
