Everything you ever wanted to know about pipelines but were afraid to ask (Part II)


In the previous post you learned some concepts of pipelines so by now you should have a good idea of what Pipelines are. In this part we are going to talk about creating\developing Pipeline components. Before doing this let cover some more basics.


Pipeline Editor


Pipeline Editor is a GUI tool that allows you to modify and create new pipeline files for more details see links bellow in the Commerce Server help documentation.



Pipeline Register (PipeReg.exe)


Commerce Server 2007 includes the Pipeline Component Registration Tool for efficient registration of creating custom pipeline components. Typically, registration would be done using the DllRegisterServer entry-point routine, as is done in C++.


The Pipeline Component Registration Tool simplifies the registration process for none C++ componenets by registering pipeline components for stage affinity and then exporting the data to a Registry Script (.reg) file so it can be deployed on other servers.


The PipeReg.exe tool is located under C:\Program Files\Microsoft Commerce Server 2007\Tools if you installed Commerce Server in its default location.


We will go over this tool more once we start developing pipeline components using .NET.


Commerce Server SDK and Pipelines


Under the Commerce Server SDK folder you will find three folders of interest when developing pipelines.


DumpOrder


A sample Microsoft Visual Basic script, DumpOrder.vbs, is supplied in the C:\Program Files\Microsoft Commerce Server 2007\Sdk\Samples\DumpOrder subdirectory if Commerce Server is installed in its default location. When it runs, the DumpOrder script writes the contents of the OrderForm object to a file. You use the DumpOrder script for debugging purposes.


To use the DumpOrder script



  1. Add the Scriptor component to any stage in the pipeline. Double-click the Scriptor component icon. In the resulting property page, click the Scriptor tab, click VBScript in the Language drop-down list, and then click File.
  2. In the File box, type the path to the DumpOrder.vbs script (or click Browse to find it).
  3. The DumpOrder.vbs script takes a file name as a parameter, so type the full path name of the output file in the Config field. For example, filename=c:\order.log.
  4. Click OK, and then save the pipeline configuration file.
  5. Stop and restart Internet Information Services (IIS), or modify the web.config file to restart the application. The DumpOrder.vbs script will write a file with the name you specified as a parameter when the Order Processing pipeline runs.

Note: Make sure to give permissions to where the log file is generated to ASP.NET account or whatever account you use for the process. Also make sure to understand the security implications of giving write\modify to a directory or file.


MinMaxShipping


The Commerce Server SDK includes a complete pipeline component developed in C#. It is called MinMaxShipping, and can be found in the following folder:


C:\Program Files\Microsoft Commerce Server 2007\Sdk\Samples\MinMaxShipping if Commerce Server is installed in its default location.


Pipeline Registration Tool Source Code


Pipeline Registration Tool Source Code under C:\Program Files\Microsoft Commerce Server 2007\Sdk\Samples\Pipeline Registration Tool Source Code folder if Commerce Server is installed in default location and constitute the source files for Pipeline Register (PipeReg.exe)  that helps you register pipeline components for stage affinity. It allows you to register the pipeline component on the computer on which the tool is being run, and it allows you to export the data to a registry script (.reg extension) for deployment to other servers.


A list of out of the box Commerce Server Pipeline Components


The following is a list of out of the box pipeline components that can be used to create custom pipeline files. Remember in the first part we went over some sample files that comes with Commerce Server and are to be used as references. Based on your business rules you can create your own custom pipelines.



























































































































































































Pipeline Component Name


Description


AddAttachments


Adds attachments to a direct mail message.


CheckInventory


Checks the inventory status for items in the customer’s basket.


ComposeDMLMessage


Creates the body of a direct mail message.


CreateUPMCookie


Collects cookie data to pass to an Active Server Pages (ASP) page creating a personalized direct mail message.


CSFAdvertisingNeedOfDelivery


Makes sure that advertisements are selected often enough to meet business commitments.


CSFEvalTargetGroups


Used to target particular groups for advertising, content selection, and discounts. This component evaluates a list of expressions for each item in a list and adjusts item selection scores.


CSFFilterContent


Applies provided filters to a content list in a Content Selection pipeline (CSP).


CSFFormatTemplate


Merges data from selected items and their associated templates to form formatted strings. These are usually HTML strings ready for display on the page.


CSFHistoryPenalty


Applies penalties to content items based on how recently they have been selected.


CSFIISAppendToLog


Records information about content selection results in the QueryString field of the Internet Information Services (IIS) log file.


CSFInitPipeline


Used at the beginning of a Content Selection pipeline (CSP) to initialize the values needed in the rest of the pipeline.


CSFLoadAdvertisements


Loads advertising data for the Content Selection Framework through the CacheManager object.


CSFLoadDiscounts


Loads discount data for the Content Selection Framework through the CacheManager object.


CSFLoadHistory


Retrieves a history string from a user profile, an item in the ASP session collection, or a cookie.


CSFRecordEvent


Records event delta counts for the selected content items (the winners) in the Performance dictionary.


CSFRecordHistory


Records the identifiers of the winning content items in the history list string.


CSFSaveHistory


Records the history string in one of three locations: the User Profile object, the Active Server Pages (ASP) session collection, or the Hypertext Transfer Protocol (HTTP) cookie.


CSFScoreDiscounts


Adjusts scores of discount items to promote discounts most relevant to the current user.


CSFSelectWinners


Selects items based on their final scores in the Content Selection pipeline.


CSFWriteEvents


Used with the CacheManager object to log Content Selection Framework events.


DefaultHandlingCy


Used as placeholder handling components. They initialize the total handling for the order to zero (0).


DefaultItemPriceCy


Set the regular price of an item to the list price returned by pipeline components such as QueryCatalogInfo or QueryProdInfoADO.


DefaultOrderSubTotalCy


Calculates the subtotal for the order.


DefaultPayment


Assigns a default value to the payment authorization code. This component is usually replaced by a third-party payment component.


DefaultShippingCy


Placeholder shipping components. They set the shipping cost to zero (0).


DefaultTaxCy


Set the tax fields of the order to zero (0). These components are usually replaced by third-party tax components.


DefaultTotalCy


Calculates the order total from the subtotals.


OrderDiscount


Applies discounts to the shopping basket.


OrderLevelDiscountApply


Applies order-level subtotal discounts.


PaymentMethodRouter


Processes payments on an order.


POtoFile


Sends a purchase order or other textual data to a file.


QueryCatalogInfo


Retrieves catalog information for every item in the order form.


RecordPromoCodes


Records the promotion codes that have been applied to the order and adds promotional discounts to the list of discounts for the order form.


RedeemPromoCodes


Validates promotion codes and marks them as redeemed.


RequiredHandlingCy


Verifies that the handling total is present in the order.


RequiredItemAdjustPriceCy


Ensures that the current price for each item in the order is set and that it has not been changed since the last time the basket was viewed.


RequiredItemPriceCy


Verifies that each item in the order form has a regular price assigned to it.


RequiredOrderAdjustPriceCy


Adjusts the total cost of each item in the order after applying any discounts.


RequiredOrderCheck


Verifies that the order form contains at least one item.


RequiredOrderInitCy


Initializes the payment authorization code, order ID, totals, and subtotals in the order form.


RequiredOrderSubtotalCy


Ensures that a subtotal exists on the order form.


RequiredPayment


Verifies that the payment authorization code exists in the order form (is not NULL).


RequiredProdInfo


Deletes items flagged for deletion by a previous Product Info stage component.


RequiredShippingCy


Verifies that the shipping total has been set.


RequiredTaxCy


Verifies that the total tax and tax included appear in the order form.


RequiredTotalCy


Verifies that values on a page have not been tampered with or reset. The components compare values in the order against values in the _Verify_With dictionary.


ReservePromoCodes


Validates and reserves promotion codes.


Scriptor


Executes a script in a Microsoft ActiveX scripting language.


SendPrecomposedMessage


Sends a direct mail message composed in earlier stages in the pipeline.


SendSMTP


Sends an e-mail message to a specified recipient.


ShippingDiscountAdjust


Applies shipping discounts to the order.


ShippingMethodRouter


The ShippingMethodRouter pipeline component looks up and runs the pipeline components that calculate shipping costs for individual shipments, and then calculates the shipping total for the entire order.


Splitter


Divides the order into groups based on a list of item keys. Frequently used to divide the order into multiple shipments.


StepwiseShipping


Calculates shipping costs for shipments in the multiple-shipment shipping architecture. This component is never directly used in a pipeline; it is run by the ShippingMethodRouter pipeline component.


ThrottleDMLPerformance


Controls the speed with which direct mail messages are processed by the pipeline.


UpdateInventory


Checks the inventory status for an array of stock-keeping units (SKUs) and updates the SKU quantities in the Inventory System.


ValidateCCNumber


Checks the expiration date of a credit card and performs a checksum test on the number.


VerifyMessageBody


Checks the body of a direct mail message according to the user flags.


VerifyRecipientData


Checks the data about the recipient of a direct mail message.


XMLTracer


A troubleshooting and debugging tool that outputs the contents of the OrderForm object to a log file during any stage of pipeline execution.


Developing a custom Pipeline Component


Finally we can create pipeline components using Microsoft Visual C#, you can also create pipelines using Visual Basic 2005 and C++. So lets walk through a step by step guide for developing a custom pipeline component in this sample we will work with C#.



  1. Start Visual Studio 2005.
  2. On the File menu, click New, and then click Project. The New Project dialog box opens. 
  3. In the New Project dialog box, in the Project types pane, select the Windows child within the Visual C# branch of the tree, and then in the Templates pane, select the Class Library template. 
  4. In the New Project dialog box, choose an appropriate project name, location, and solution name, and then click OK. Your new class library project opens within Visual Studio 2005.
     
  5. Open the Solution Explorer. Right-click your new project and then click Properties. The multi-tabbed project properties window opens in your workspace. 
  6. In the multi-tabbed project properties window, click the Build tab. Then, in the Output section, select the Register for COM interop checkbox. With this setting, Visual Studio 2005 will automatically register your pipeline component with COM.
     
  7. In the multi-tabbed project properties window, click the Application tab. Then, in the Output section, select the Assembly Information button.

    From the Assebly Information Dialog check Make assembly COM-Visable. Specifies whether types within the assembly will be accessible to COM. Corresponds to ComVisibleAttribute.

  8. In Solution Explorer, right-click your project and then click Add Reference. The Add Reference dialog box opens.
     
  9. In the Add Reference dialog box, on the .NET tab, scroll if necessary and select Microsoft Commerce Server Interop Assembly, and then click OK. Your project now includes a reference to Microsoft.CommerceServer.Interop.






    Note


    Many pipeline components will also need to reference the assemblies Microsoft Commerce Server .NET Application Framework and Microsoft Commerce Server Shared Types Assembly. Repeat steps 7 and 8 as required.


    If you are passing Commerce Subsystem Context values or need to have access to .NET Commerce objects then you will use the Microsoft Commerce Server .NET Application Framework and Microsoft Commerce Server Shared Types Assembly to access Commerce resources. Microsoft Commerce Server Interop Assembly is required for the pipeline interfaces.


  10. In Solution Explorer, right-click the file Class1.cs and then click Rename.
  11. Type a more descriptive name for your C# source code file (make sure you retain the “.cs” file extension) and the press ENTER. When prompted about renaming all references to related code elements, click Yes. The name of your pipeline class will change as specified within your renamed C# source code file.
  12. In your renamed C# source code file, add the following using directives below the default using directives:



    using System.Runtime.InteropServices;
    using Microsoft.CommerceServer.Runtime;
    using Microsoft.CommerceServer.Interop;
    using Microsoft.CommerceServer.Interop.Orders;

To add the appropriate attributes to your pipeline component class



  1. In Visual Studio 2005, on the Tools menu, click Create GUID. The Create GUID dialog box opens.

  2. In the Create GUID dialog box, choose the Registry Format, click Copy to create a GUID and copy it to the clipboard, and then click Exit to dismiss the tool.

  3. Add the following two bolded lines immediately above your pipeline class declaration in your C# source code file, pasting the GUID created in step 2 from the clipboard. 









    Important

    The GUID shown below and throughout this topic is an example. You must replace it with a unique GUID of your own.

     

    [ComVisible(true)]


    [GuidAttribute(“D452305E-50C9-4031-BC94-6839BE6066EE”)]
    public class MyPipelineClass { }


To implement your pipeline class


Type the names of the interfaces from which you want your pipeline component to inherit. It must inherit the interface IPipelineComponent, and optionally it may inherit one or both of the interfaces IPipelineComponentAdmin and IPipelineComponentDescription. For this example we are only going to implement the IPipelineComponent.





[ComVisible(true)]
[GuidAttribute(“D452305E-50C9-4031-BC94-6839BE6066EE”)]
public class MyPipelineClass : IPipelineComponent { }

Your project should look something like bellow:






using System;
using System.Collections.Generic;
using System.Text;


using System.Runtime.InteropServices;
using Microsoft.CommerceServer.Interop;
using Microsoft.CommerceServer.Runtime;
using Microsoft.CommerceServer.Interop.Orders;


namespace CustomPipeline
{
    [ComVisible(true)]
    [GuidAttribute (“5904C354-F1B8-485c-89DD-883D26BCE85D”)]
    public class Class1 : IPipelineComponent
    {
        // Status codes for pipeline components
        private const Int32 StatusSuccess = 1; // success
        private const Int32 StatusWarning = 2; // warning
        private const Int32 StatusError = 3; // error


        #region IPipelineComponent Members
        void IPipelineComponent.EnableDesign(int fEnable){}


        int IPipelineComponent.Execute(object pdispOrder, object pdispContext, int lFlags)
        {
            Int32 ReturnValue = StatusSuccess;
            // TODO: add code for the pipeline
            return ReturnValue;
        }
        #endregion
    }
}


Compile your project and after a successful build we will register your new pipeline component.

Register your Pipeline


You use PipeReg.exe tool to register your newly created pipeline component. We discussed the tool earlier also the source code for the tool is available under the SDK folder. PipeReg.exe can be located under c:\Program Files\Microsoft Commerce Server 2007\Tools\ if Commerce Server was installed in its default location.



  1. Launch the PipeReg.exe EXE.

  2. From the Pipeline Component Registration Wizard [Step 1 of 3] dialog Select Select component Type Library. Next specify the path by selecting the Path button with ellipses. When you compiled your pipeline component Visual Studio generated a Type Library file be sure to select that file then Select Next.

  3. From the Pipeline Component Registration Wizard [Step 2 of 3] dialog Select All Stages from Available Categories List Box then Select the > Button. This will add our newly component to the All Stages category. Here you can also create your own custom stages.

  4. From the Pipeline Component Registration Wizard [Step 2 of 3] dialog Select Register and export data. Make sure to note the location and name of the reg file. Then Select Next. Then Select Close after viewing the summary page.

Test your Pipeline


We will test our pipeline by loading it into a pcf file to make sure that the registration succeeded.



  1. Launch the Pipeline Editor.
  2. From the File menu Select New. Then from the Choose a Pipeline Template Select the OK button to choose the default template.

  3. After the Pipeline Template has been loaded right click on any stage and then Select Insert Component.

  4. From the Choose a Component dialog Select All from the Stages Drop Down. Then navigate until you find your component then Select OK.

  5. Congratulations you have successfully created and tested your .NET pipeline component.

Making your component ready for production deployment


To deploy your new component Copy the assembly to the target computer and register the component using Regasm.exe. You must either deploy the component into the global assembly cache (GAC) by using Gacutil.exe or you may use Regasm.exe with the /codebase option to deploy your new pipeline component without putting it in the GAC. I would recommend your strongly sign your new component and add them into the GAC.


Summary


In this part your learned some more concepts on how to work with Pipeline Editor and Pipeline Registration when creating a custom pipeline component. You also walked through a step by step guide of creating a C# pipeline component and testing it to make sure it works.
In the next part of this ongoing post we will look deeper into the Interfaces of the pipeline components as well as way to debug your pipeline components. I will also show you how you can monitor the performance of your pipelines during execution.

Comments (12)

  1. LaptopHeaven says:

    Using the pipeline seems a little antiquated.  Why couldn’t you just write .NET classes with execute inside the ASP.NET Commerece Server site?

  2. MSDNArchive says:

    The Pipeline Processing is a pattern (Linear pipeline pattern) used in which data flows through a sequence of task or stages. So if your component needed to be executed in a specific stage with the current Pipeline object there is no way to separate this. Meaning perform stage 1 then stop run your .NET object then continue with the rest of stages. So this is why you have to create a pipeline component to operate within the pipeline object.

    The Pipeline line object abstract the work needing to be performed. The pipeline object will load the PCF file then go through the stages and execute the execute method (the simple explanation).

    You can create your own Work flow to do the same thing in ASP.NET process that the Pipeline object is doing but you have to do a lot of work read the pipeline file go through each stage call the execute method.

    There is another thing that you need to take into account. I will explain in my next post and that is before a pipeline is executed. The Pipeline adapter maps the .NET OrderForm into the COM OrderForm. An XML file holds this mapping, so stay tuned until the next post.

    I hope that I have answered your question.

    -Max

  3. Philip says:

    Im looking forward to the next part. This is very informative and well writen!

    Best Regards

    Philip

  4. Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server Pipelines system.

  5. Max Akbar says:

    So far you if you have followed this on going Pipeline posts, you should be very familiar with how to…

  6. Biztalk 2006 says:

    Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server…

  7. As previously discussed the pipeline system enables sequential workflow processing. It is a COM-based…

  8. Josephine says:

    Hi Max. Thanks for writing such a good article on pipelines. I’m new to commerce server and is thinking to create new pipelines. Unfortunately, I always get this error "Component Execution failed for component[0x3]  hr: 0x80040154

    ProgID: CustomPipeline.Class1

    Could not create Component CustomPipeline.Class1.  This may be a component registration issue." when i run the shopping cart. Do you have any idea how would that happen?! Thanks very much!

  9. MSDNArchive says:

    Hello Josephine,

    There is not a lot of context here for me to help you. Use this blogs email and send me an email. I will need to look at your code and some other details about exactly what it is you are doing. Then I can help you.

    -Max