How to Develop BizTalk Custom Pipeline Components – Part1


Receive and send pipelines in BizTalk are used to perform a range of processing and operations on messages. They can –


 


1.      Encrypt and decrypt messages


2.      Sign and verify digitally signed messages


3.      Validate message against schema


4.      Deal with promoted properties used for content based routing


5.      Disassemble or break single message into multiple


6.      Wrap messages with header and footer


 


Pipelines contain stages and each stage can hold more than one pipeline component. Pipelines and pipeline components present out of box can do most of the tasks for you. But sometime specific message processing or massaging requirements encourage developers to develop custom pipeline components. Current series is focused on describing development of custom pipeline components.


 


There are multiple categories of pipeline components each doing specific job when employed on their predefined stages. These categories are –


 
























Pipeline Components


Stages Employed


Tasks


General Component


Decode, Encode, Pre-assemble, Resolve Party or Validate


Take one message process message and produce zero or one message


Disassemble Component


Disassemble


Split message, promote custom properties


Assemble Component


Assemble


Used to wrap message with head or trailer or both


Probe Component


This is not an independent component. Any pipeline component can implement the IProbeMessage interface if it must support message probing functionality.


Enables the component to check the beginning part of the message


 


During development, each of these components implements particular interfaces and execute custom written job (code) on messages. I will walk through development of each of these components with sample code.


 


Developing General Pipeline Component


As stated earlier, pipeline components implement certain interfaces. General pipeline component implements following interfaces –


 


a.      IBaseComponent Interface


b.      IComponentUI Interface


c.      IComponent Interface


d.      IPersistPropertyBag (Optional. Required when pipeline design time properties are to be defined)


 


IBaseComponent Interface


All pipelines implement this interface. Interface provides members which can be implemented to provide information about pipeline.


 
















Members


Usage


Description


Property. Used to specify small description about pipeline component. Description is visible on pipeline properties page at design time.


Name


Property used to specify name of pipeline component. Name is visible on pipeline properties page at design time.


Version


Property used to specify version (example 1.0.0.0) of pipeline component. Visible on pipeline properties page at design time.


 


IComponentUI Interface


All pipeline components must implement this interface. Members of this interface provide design time support.


 













Members


Usage


Icon


Property used to provide icon associated with pipeline component.


Validate


Method. This is called by pipeline designer before pipeline compilation to verify that all configuration properties are correctly set.


 


IComponent Interface


This is core interface. Member of this interface is implemented for specific message processing/massaging.


 










Members


Usage


Execute


Method. Does specific processing/massaging in inbound message and produces output message to be forwarded to next stages of pipeline or message box.


 


IPersistPropertyBag


Interface provides members which can be used to save and load pipeline design time properties with property bag. This interface is implemented only when pipeline’s design time properties are to be defined.


 



















Members


Usage


GetClassID


Method. Retrieves the component's globally unique identifying value.


InitNew


Method. Initializes any objects needed by the component to use the persisted properties.


Load


Method. Used to load property from property bag.


Save


Method. Used to save property to property bag.


 


Enough theory. This is time for some coding.


 


Code Walk Though with Sample


 


I am going to create a sample general pipeline component. Scenario is very simple. Whenever, I receive a message, I want to change namespace of message in pipeline component. Namespace to be changed can be configured in design time property of pipeline.


 


If I get a message like following –


 


<Input xmlns:ns0="http://MyNamespace">


                <FirstName>Peter</FirstName>


<LastName>Decosta</LastName>


  </Input>


 


And if changed namespace property is set to http://MyNewNamespace, then output message (which goes to next stages of pipeline and message box) would become –


 


<Input xmlns:ns0="http://MyNewNamespace">


                <FirstName>Peter</FirstName>


<LastName>Decosta</LastName>


  </Input>


 


I am using VS 2005 for development.


 


1.      Create a class library project say SampleGenPipelineComponent.proj. When project is created, rename Class1.cs to SampleGenComponent.cs.


 


2.      Configure strong name key file in project property so that assembly can be signed.


 


3.      Add following reference


 


Microsoft.BizTalk.Pipeline.dll


 


References can be found in folder “C:\Program Files\Microsoft BizTalk Server 2006” depending on location of BTS installation.


 


4.      Add following namespaces.


 


using System.Xml;


using System.IO;


using Microsoft.BizTalk.Message.Interop;


using Microsoft.BizTalk.Component.Interop;


 


5.      Implement following interfaces-


 


IBaseComponent,  IComponentUI,  IComponent, IPersistPropertyBag


 


6.      Add following attributes to pipeline component class


 


     [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]


    [ComponentCategory(CategoryTypes.CATID_Decoder )]


    [System.Runtime.InteropServices.Guid("9d0e4103-4cce-4536-83fa-4a5040674ad6")]  


public class SampleGenComponent :


                                        IBaseComponent,


                                        IComponentUI,


                                        IComponent,


                                        IPersistPropertyBag


                  


General components can be used in various stages – Decode, Encode, Validate etc.  Component Category attribute is used to limit use of general pipeline to restricted stages. In my example, I am restricting my pipeline to be used in Decode stage only. This concept is called “Stage Affinity”. Please read following MSDN link to learn more about this –


http://msdn2.microsoft.com/en-us/library/ms964541.aspx


    


7.      Implement members of IBaseComponent interface.


 


        public string Description


        {


            get


            {


                    return "Pipeline component used to change namespace of message";


            }


        }


        public string Name


        {


            get


            {


                return "SampleGenPipelineComponent";


            }


        }


        public string Version


        {


            get


            {


                return "1.0.0.0";


            }


        }


      


Member properties are overriden to provide description, name and version of pipeline component.


 


8.      Implement members of IComponentUI interface.


 


    public IntPtr Icon


        {


            get


            {


                return new System.IntPtr();


            }


        }


 


     public System.Collections.IEnumerator Validate(object projectSystem)


        {


            return null;


        }


 


I have kept things simple and have not provided any pipeline design time icon and validation logic. In actuall implementation, you can add an icon in piepline resouce file and point that in impelemtation. Similarily, you can also verify design time configuration/properties in Validate method and can return all inconsistancies in collection of errors.


 


9.       Implement members of IPersistPropertyBag interface.


 


 


        private string  _NewNameSpace;


        public string NewNameSpace


        {


            get { return _NewNameSpace; }


            set { _NewNameSpace = value; }


        }


 


        public void GetClassID(out Guid classID)


        {


            classID = new Guid("655B591F-8994-4e52-8ECD-2D7E8E78B25C");


        }


       


        public void InitNew()


        {


 


        }


       


        public void Load(IPropertyBag propertyBag, int errorLog)


        {


            object val = null;


            try


            {


                propertyBag.Read("NewNameSpace", out val, 0);


            }


            catch (Exception ex)


            {


                throw new ApplicationException("Error reading propertybag: " + ex.Message);


            }


            if (val != null)


                             _NewNameSpace = (string)val;


                             else


                        _NewNameSpace = "http://AnonymusURL";       


        }


 


        public void Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties)


        {


            object val = (object)_NewNameSpace;


            propertyBag.Write("NewNameSpace", ref val);           


        }


 


First of all, we created “NewNameSpace” property which is used to configure new namespace name at design time.


 


“Load” method loads value from property bag to pipeline property. “Save” method saves value to property bag from pipeline property. “GetClassID” returns component's unique identifying value in terms of GUID. “InitNew” is used to initialize object to be persisted in component properties. I have not implemented it because I don’t require.


 


10.  Implement members of IComponent interface.


 


        public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)


        {


            IBaseMessagePart bodyPart = pInMsg.BodyPart;


            StringBuilder outputMessageText=null;


            string systemPropertiesNamespace = @"http://schemas.microsoft.com/BizTalk/2003/system-properties";


            string messageType = "";


 


            if (bodyPart != null)


            {


                Stream originalStream = bodyPart.GetOriginalDataStream();


                if (originalStream != null)


                {


 


                    XmlDocument xdoc = new XmlDocument();


                    xdoc.Load(originalStream);


                    XmlElement root = xdoc.DocumentElement;


 


                    messageType = this.NewNameSpace + "#" + root.Name;


 


                    outputMessageText = new StringBuilder();


       outputMessageText.Append("<" + root.Name + "  xmlns:ns0='" +   this.NewNameSpace + "'>");


                    outputMessageText.Append(root.InnerXml);


                    outputMessageText.Append("</" + root.Name + ">");


 


                    byte[] outBytes = System.Text.Encoding.ASCII.GetBytes(outputMessageText.ToString()); 


 


                    MemoryStream memStream = new MemoryStream();


                    memStream.Write(outBytes, 0, outBytes.Length);


                    memStream.Position = 0;


 


                    bodyPart.Data = memStream;


                    pContext.ResourceTracker.AddResource(memStream);


                  }


               }


            pInMsg.Context.Promote("MessageType", systemPropertiesNamespace, messageType);


            return pInMsg;                              


        }


 


This is core of implementation. Code flow is –


 


-         First read body part of message … = pInMsg.BodyPart;”


 


-         Then read message content (XML) in stream “… = bodyPart.GetOriginalDataStream();”


 


-         Change namespace of XML content “outputMessageText”


 


-         Create a stream of new XML (with namespace)


 


-         Assign stream back to message body part “bodyPart.Data = memStream;“


 


-         Since namespace has changed, we need to update message type context property otherwise there is a fair chance of message routing failure. “pInMsg.Context.Promote("MessageType",…”. To understand importance of MessageType, please refer “Message Processing” section in link - http://msdn2.microsoft.com/en-us/library/ms935116.aspx


 


-         Return message so that it can be consumed by next stages of pipeline and/or message box itself.


 


11.  Coding is complete. Build the project.


 


Deploy Component and Use


To deploy pipeline component, Copy SampleGenPipelineComponent.dll to “C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components”.


Pipeline component is now ready to use. In BTS receive pipeline project, add this pipeline component dll in toolbar and then use in decode stage. You will find that you can set value of “NewNameSpace” property at pipeline design time.


Summary


This was a sample general pipeline component which can be deployed to decode stage. Similarly, you can create other general pipeline components for requirement specific processing and data massaging. Hope this entry was useful. I am also attaching code zip file (SampleGenPipelineComponent.zip) for any further reference.


 

SampleGenPipelineComponent.zip

Comments (12)

  1. Brad Friedlander says:

    This solution works well for small and moderate sized messages. Do you have any alternative suggestions when the message size is large (i.e., many MBs in size)?

  2. brajens says:

    With large message, it is not advisable to use XMLDocument. You can use other XML APIs, XmlReader, XmlWritter or any other stream based Xml modification API. I will do search on this and get back to you.

  3. AK says:

    Hello,

    How would I get the guid of the class to be used in GetClassID ?

    thanks

    AK

  4. to AK…

    You may create him with "Create Guid" tool in Visual Studio…

    For this in VS -> Tools -> Create GUID-> New GUID(in Registry Format) and copy him to your project.

    or  

    System.Guid myGuid = new Guid();  in C# code….

    Vadim

  5. RS says:

    I have a scenario wherein I get flat file as input . I need to access and modify few contents of the ip file using validate stage of custom pipeline.How do I access the fields(actual data) of my ip file within th custom pipeline Execute() method?

  6. brajens says:

    Hi RS

    If you check sample given with blog, you can get message body content as stream –

    Stream originalStream = bodyPart.GetOriginalDataStream();

    if  this stream is XML based data then you can use .net DOM APIs or XMLReader to get required xml elements/fields.

    if stream is non-xml data then you need to do looping of records and record parsing to get required field data.

  7. Bruno Spinelli says:

    It’s recommended to use VirtualStream to manipulate message streams in pipeline processing. HIH.

    Bruno Spinelli.

  8. oops says:

    Fantastic Effort. But one question. I have a situation in which I am getting the Customer ID in the incoming XML message, I have to check it against the values stored in the table of a database. Can I do that in the Execute Message?

  9. Subbu says:

    I am getting a error when i am trying to use this component

  10. AK says:

    How should i dispose/close the memory stream used in this sample. if i do so, i don’t get the changed message when debugging the pipeline?

  11. Gus says:

    Hi, excellent post. I have followed the example, signed the dll, dropped it in the Pipeline Components folder, I go to my Bisztalk Server project, and when I try to add the dll to my Toolbox, it says that there are no components that can be placed in the toolbox. My component class is public, so I don’t know what can be causing the problem.

    Any ideas?

  12. wvanhuffel says:

    I like it! thanks very much for writing this article – I will be referenceing it as soon as I understand how to create pipelines in BTS – regards, Van (wvhuffel)

Skip to main content