Everything you ever wanted to know about pipelines but were afraid to ask (Part I)


What are Pipelines


Commerce Server pipeline is a framework that links one or more components and executes them in sequence. The Pipeline framework provides stages to abstract one or more categories of work. The Pipeline Stage also determine the sequence in which each category of work is performed. Each Pipeline Component is COM object that is put inserted into a stage and is executed in sequence by the Pipeline Object. Pipelines are linked and it’s configuration stored in files Pipeline configuration files (.pcf).


Pipeline Execution


The following diagram displays the Basket Pipeline and the execution starts with the Product Info Stage and continues down to the last Stage Order Subtotal. You can add or delete Stages as well as move the sequence of a particular Stage.


 


Figure 1 Commerce Server Pipeline


Pipeline Properties


Use the Pipeline Properties dialog box to change the name of the pipeline, to specify its transaction compatibility, to specify its currency mode, or to save notes about the pipeline.



Figure 2 Pipeline Property


The following is a description of each property for the UI above.






















Use this


To do this 


Label


Type the name that the Pipeline Editor displays for the pipeline.


Transaction


Select which pipeline object that you can use the pipeline configuration file (.pcf) with. The valid options are as follows:


·         Any pipeline. You can use the configuration file together with an OrderPipeline object.


·         Requires transacted pipeline. You can use the configuration file together with transacted pipeline objects (PooledTxPipeline, MtsTxPipeline).


·         Requires non-transacted pipeline. You can use the configuration file together with non-transacted pipeline objects (PooledPipeline, MtsPipeline).


Supply a currency LCID


Specify a locale ID (LCID) for currency handling. A value of zero (0) indicates to use the default locale ID of the server. The default currency LCID is zero (0).


Supply the number of decimal places used for currency rounding


Specify how many digits of precision to use for the currency. For example, you use two decimal places for United States pennies. Commerce Server uses this value for rounding purposes, and to determine the scaling factor to use in legacy currency keys.


Description


Type descriptive comments about the pipeline.


Pipeline Stages


The following diagram displays the Product Info Stage which consists of three Pipeline components. You can add or delete components using the Pipeline Editor. You can also move the individual components sequence.



Figure 3 Product Info Stage


Stage Properties


Use the Stage Properties dialog box to change the properties assigned to a pipeline stage.



Figure 4 Stage Properties                                                                                                                                                    


The following is a description of each property for the UI above.

























Use this


To do this 


Label


Specify the name that the Pipeline Editor displays for the stage.


Description


Type descriptive comments about the stage.


GUID


View the GUID that identifies the component on any computer. This ID never changes.


Mode


A value of zero (0) prevents the stage from running.


Error level


Specify the error level for the stage.


Note


When any component in the stage generates an error above the level specified in the Error level box, the stage skips the remaining components in the stage, and logs the event in the log file.


 


The pipeline maintains the maximum error level encountered by any component in the pipeline. It uses the Error Level setting on the stage to compare this maximum error level and only runs components in the stage if the current error level is less than or equal to the error level limit specified for the component.


Pipeline Component


A component is a Component Object Model (COM) object designed to perform some operation on a dictionary -- a collection of name/value pairs -- that is a different representation of a runtime object. This can be a dictionary that represents an OrderForm object or a dictionary that represents another type of object.


 


Figure 5 Query Catalog Info Pipeline Component


Components Properties


Use the Component Properties dialog box to change the name of a pipeline component, to view the class ID or Program ID of a component, and to save notes about a component. Use the Component Properties dialog box, Values Read and Written tab, to view the values that the component reads from and writes to the order form and the values it reads from the Context dictionary object. For descriptions of each value read and written, access the corresponding component from the Pipeline Component Reference. These two “Component Properties” and “Values Read and Written” are common to all Pipeline components.



Figure 6 Components Properties


The following is a description of each property for the UI above.






















Use this


To do this 


Label


Specify the name that the Pipeline Editor displays for the component.


Class ID


View the GUID that identifies the component on any computer. This ID never changes.


Program ID


View the easily recognizable ID for the component. This ID never changes.


Required


Check to make sure that the pipeline component is not removed from the pipeline.


Description


Type descriptive comments about the component.


 



Figure 7 Component Values Read and Written


The following is a description of each property for the UI above.


Note: The Pipeline Editor cannot display values read and written for components that do not support the IPipelineComponent Interface. We will discuss the Interfaces that the Pipeline components implement later on.
















Use this


To do this 


Values Read


View the values that the component reads from the order form before it performs its function.


Values Written


View the values that the component writes to the order form or other dictionaries after it completes its function.


Context Values Read


View the values that the component reads from the Context dictionary object. The Context dictionary object is passed as a parameter to the Execute method of the pipeline.


 


There are however some out of the box components that need additional configuration or other information, this data is displayed on additional property pages.




















































Properties Dialog Box Tab Name


Description


Eval Expressions Tab


Determines how the CSFEvalTargetGroups component evaluates targeting expressions for advertising, content selection, and discounts.


Execution Mode Tab


Determines how the PaymentMethodRouter component processes credit cards and whether to validate payment totals.


History Penalty Tab


Determines how the CSFHistoryPenalty component applies penalties to content items based on how recently they have been selected.


Load History Tab


Determines where the CSFLoadHistory component retrieves the history string.


Order Discount Tab


Determines how the OrderDiscount component applies discounts scheduled by the Marketing Manager to the shopping basket.


Order Level Discount Tab


Determines how the OrderLevelDiscountApply component applies order level subtotal discounts.


Payment Router Tab


Determine the behavior of the PaymentMethodRouter pipeline component when processing credit cards and validating payment totals.


PO to File Tab


Determines how the POtoFile component retrieves and sends a purchase order or other textual data to a file.


Record History Tab


Determines how many items the CSFRecordHistory component saves in the history list string, which is used to contain the identifiers of winning content items.


Save History Tab


Determines where the CSFSaveHistory component saves the history string.


Scriptor Tab


Stores the script to be run by the Scriptor component.


Select Winners Tab


Determines how the CSFSelectWinners component displays house ads.


Send SMTP Tab


Determines where the SendSMTP component sends an e-mail message and the information sent.


Splitter Tab


Determines how the Splitter component divides an order into groups based on a list of item keys.


Test SendSMTP Tab


Tests the configuration of the SendSMTP component by sending a test e-mail message.


 


Commerce Server provides five objects to execute pipelines:


1.       MtsPipeline


Use this object to run a non-transacted Microsoft Commerce Server pipeline.


2.       MtsTxPipeline


Use this object to run a transacted Microsoft Commerce Server pipeline configuration.


3.       PooledPipeline


Use this object to run a Microsoft Commerce Server pipeline configuration file. The PooledPipeline object uses COM+ object pooling to greatly increase the loading speed of pipelines, where the contained pipeline components support object pooling.


4.       PooledTxPipeline


Use this object to execute a Microsoft Commerce Server pipeline. Use in place of the PooledPipeline object where you need transactions. The PooledTxPipeline object uses COM+ object pooling to increase the loading speed of pipelines.


5.       OrderPipeline


Use this object to run a Microsoft Commerce Server Content Selection pipeline. To create an Order Processing pipeline (OPP), use a PooledPipeline, PooledTxPipeline, MtxPipeline, or MtsTxPipeline object.


The MtsPipeline and MtxTxPipeline is recommended for development only. All other Pipelines are recommended for production. The reason for this is that the MtsPipeline and MtxTxPipeline objects do not load the configuration into memory and makes it easier to develop rather than stopping and starting IIS every time a change occurs.


Note: MicroPipe is no longer supported in Commerce Server 2007. In earlier version of the Commerce Server you could only execute the Pipeline component.


By default Commerce Server has the following  Pipeline files:


1.       Advertising.pcf


2.       Basket.pcf


3.       Chckout.pcf


4.       Discounts.pcf


5.       DMLPipe.pcf


6.       Product.pcf


7.       RecordEvent.pcf


8.       Total.pcf


When unpacking the CSharp site you will also get the above Pipeline files. These Pipeline files can be used as template to start a new Commerce Site.


Note: Commerce Server 2007 does not have Pipeline templates.


Pipelines in a nutshell


So to summarize everything, Pipeline framework consists of COM objects that are linked and stored in pcf file format. You can further categorize pipelines into stages. Pipelines are executed by the Pipeline object and configured by Pipeline Editor. Pipelines can be executed transactional and be pooled for performance. Commerce Server has a dozen or more Pipeline components and several default pcf files containing one or more Pipeline components. You can also create your own Pipeline components.


How are Pipelines used in Commerce Server


Commerce Server has three types of Pipelines:


1.       Order Processing


The Order Processing pipelines (OPP) are software infrastructures that link several components and run them in sequence on an OrderForm or Dictionary object. The Order processing Pipeline implement aspects of business-to-consumer and business-to-business checkout transactions such as discounts, Inventory checking, Product Catalog checking etc…


2.       Content Selection Framework


The Content Selection Framework (CSF) is a platform-level framework for building targeted content delivery applications on the Internet. The targeted advertising and discount features of Microsoft Commerce Server are built by using the CSF. The framework provides the means for building new types of content delivery applications and makes it easy to customize and extend the advertising and discount delivery systems.


3.       Direct Mailer


The Direct Mailer pipeline runs components that screen recipients and personalize e-mail, and then sends the personalized e-mail.


You can add third-party or custom components to the Commerce Server pipeline. Those components need implement only the standard pipeline component interfaces. They also, however, need to understand the dictionary keys that components in the pipeline use.


How Pipelines are executed


To execute a pipeline, you add the name of the pipe to the pipelines node for your site in the site Web.config file, see example bellow.






<pipelines siteName="CSharpSite">


  <pipeline name="basket" path="pipelines\basket.pcf" transacted="true" type="OrderPipeline" loggingEnabled="false"/>


</pipelines>


 


Each pipeline node must have the following attributes:


·         Name - The name that you will reference in your application when executing the pipeline.


·         Path - The relative path of the pipeline configuration file (.pcf).


·         Transacted - Set this attribute to true or false to indicate the pipeline’s transacted state.


·         Type - You can use either OrderPipeline or CSFPipeline for this attribute. You use an OrderPipeline to process orders and a CSFPipeline to display advertisements.


·         LoggingEnabled - Set to true to enable logging. When you enable logging, the log file is created in the pipelines\log directory. You must also set appropriate permissions for the current w3wp.exe process to be able to create log files.


Sample code to execute a pipeline.






PipelineInfo info1 = new PipelineInfo(pipelineName, OrderPipelineType.Basket)


Basket.RunPipeline(info1);


 


Pipeline Types available.






public enum OrderPipelineType


{


      Custom,


      Product,


      Basket,


      Total,


      Checkout,


      AcceptBasket


}


 


This enumerator provides pipeline type information to PipelineInfo to allow an object to tailor its behavior to the requirements of the particular pipeline in use, without relying on the name of the pipeline.


The AcceptBasket type is used by the LOB Adapter.


When using the Custom type, the PipelineInfo constructor will not attempt to set up the pipeline context.

























Member name


Description


AcceptBasket


Indicates an AcceptBasket pipeline (e.g. AcceptBasket.pcf).


Basket


Indicates a Basket pipeline (e.g. Basket.pcf).


Checkout


Indicates a Checkout pipeline (e.g. Checkout.pcf).


Custom


Indicates a custom pipeline type.


Product


Indicates a Product pipeline (e.g. Product.pcf).


Total


Indicates a Total pipeline (e.g. Total.pcf).


 


Summary


At this point you should have a good context of what Pipelines are and in the next part we will get a deeper understanding of Pipelines by developing and understanding Pipeline internals. Keep an eye out for part II of the “Everything you ever wanted to know about pipelines but were afraid to ask”.

Comments (22)
  1. In my last post which listed a number of great new Commerce Server posts I forgot to mention that Max Akbar [MSFT] – Product Manager (Maxter Chief) has started blogging with a vengeance!

  2. wleite says:

    Hi Max,

    I’m creating a new CheckInventory Stage, to resolve some custom needs of my client and I have a doubt.

    I need to access the Invetory Catalog of the product item but I can’t find a way to use a Current Context. The workaround I create every-time a new Context for the product, but the performance is going down.

    Do you know some way to do this?

    Thank you,

    Willian

  3. MSDN Archive says:

    The IPipelineComponent interface has two methods Execute and EnableDesign.

    The Execute method has three parameters one of them is a context (pdispContext) value. See the interface implementation below:

    // IPipelineComponent

    [ComImport, Guid(“CA215520-9A2F-11D0-B86C-00C04FD7A0F9”), InterfaceType((short) 1)]
    public interface IPipelineComponent
    {
         [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
         int Execute([In, MarshalAs(UnmanagedType.IDispatch)] object pdispOrder, [In, MarshalAs(UnmanagedType.IDispatch)] object pdispContext, [In] int lFlags);
         [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
         void EnableDesign([In] int fEnable);
    }

    // IPipelineComponent

    You can pass the context using the pipelineinfo object in your Execute Method get the value from pdispContext.

    Good luck,
    -Max

  4. The StarterSite CTP is now available for download. This should help and get everyone started on a solid…

  5. wleite says:

    Hi Max,

    I’m trying what you suggest but I still can’t get the Catalog Context (and Inventory Context) inside the Pipeline.

    <Code>

    public int Execute(object pdispOrder, object pDispContext, int lFlags)

    {

     …

     

     Context = (IDictionary)pdispContext;

     

     …

    </Code>  

    For this pDispContext Object I found this Keys : Evaluator, Language, CatalogConnectionString, MessageManager, CacheManager, CatalogCacheConfiguration, ProfileService, CommerceResources and CacheName.

    None of them have the Catalog Context, so I tried to found where can I pass, in RunPipeline, the Catalog Context but I can’t find it.

    To run my pipelines I’m using the BasketHelper from TestStore.

     

    <Code>

    public PipelineExecutionResult RunBasketPipeline(string pipelineName)

    {

       if (String.IsNullOrEmpty(pipelineName))

       {

           pipelineName = BasketHelper.DefaultBasketPipelineName;

       }

       using (PipelineInfo pipeline = new PipelineInfo(pipelineName, OrderPipelineType.Basket))

       {

           return this.RunPipeline(pipelineName, OrderPipelineType.Basket);

       }

    }

    public virtual PipelineExecutionResult RunPipeline(PipelineInfo pipelineInfo);

    </Code>

    As you can see I not have any other parameters to put the Catalog Context. In the last Commerce Server Architecture Deep Dive WebCast they tell me that the Default CheckInventory and UpdateInventory call your own StoredProcedures in DataBase to do the Check and Update Inventory.

    For Workaround the problem I’m using this function:

    <Code>

    private InventoryContext CreateInventoryContext()

    {

       // Create a CatalogSiteAgent to connect to the database.

       CatalogSiteAgent catalogSiteAgent = new CatalogSiteAgent();

       catalogSiteAgent.SiteName = MyConstants.ConfigSiteName;

       catalogSiteAgent.IgnoreInventorySystem = false;

       // Create a CatalogContext object.

       CatalogContext catalogContext = CatalogContext.Create(catalogSiteAgent);

       

       // Get the InventoryContext property.

       InventoryContext inventoryContext = catalogContext.InventoryContext;

       return inventoryContext;

    }

    </Code>

    Do you have any ideas?

    Thank you,

    Willian Leite

  6. MSDN Archive says:

    Hi Willian,

    You are so very close good job. I am about to post part III of this ongoing post and has the sample code you are looking for :).

    -Max

  7. Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server Pipelines system.

  8. May says:

    runnig the checkout pipeline I get this error:

    error Component Execution failed for component[0x1] hr: 0x80004002 ProgID: Commerce.UpdateInventory Unable to cast object of type ‘System.DBNull’ to type ‘Microsoft.CommerceServer.Runtime.Configuration.CommerceResourceCollection’

    How can I debug this?Is is configuration or registery problem or is it sending bad data to pileline?

    Thanks,

    May

  9. MSDN Archive says:

    Check to make sure you have an inventory resource? If you are not using the Inventory system remove the Inventory pipelines.

    Also check the eventlog to see if you have anything there.

    Good luck,

    -Max

  10. Biztalk 2006 says:

    Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server…

  11. As previously discussed the pipeline system enables sequential workflow processing. It is a COM-based…

  12. Chuck says:

    Great blog entry.  

    You state "The MtsPipeline and MtxTxPipeline is recommended for development only."  However, nowhere can I find how to configure/control what object is used to execute a pipeline.  In fact I cannot find information about a Commerce Server install relates to COM+ at all.  From reading the product documentation, I gather that the installation wizard creates a COM+ application for the CS instance, but no further information is detailed.  Can you elaborate on this and/or provide references to material that explain this.

  13. MSDN Archive says:

    You can find information under the help files take a look here:

    Managing Pipeline Transactions  

    http://msdn2.microsoft.com/en-us/library/bb219147.aspx

    MtsPipeline Object

    http://msdn2.microsoft.com/en-us/library/aa545657.aspx

    MtsTxPipeline Object

    http://msdn2.microsoft.com/en-us/library/aa545593.aspx

    I will do more research and see how you can execute these pipelines. I know you can execute them from COM but that’s not very helpful :).

    -Max

  14. MSDN Archive says:

    I talked to some folks on the Product Group and did some research and here is what I found out.

    We only allow execution of PooledTxPipeline and PooledPipeline under ASP.NET. At least these are the only two available. You can create a COM object and execute MtsPipeline or MtsTxPipeline pipeline, but I see no value in this. We have a Pipeline Adapter that maps a COM orderform to a Managed orderform and makes no sense in creating a COM object to execute the pipeline. This is not an oversight, MtsPipeline and its Tx variant are apartment threaded and so aren’t friendly in the ASP.NET world. So these objects are there for backward compatibility and a downside is development experience.

    I hope this answers your question.

    -Max

  15. Prashant says:

    Hi Max,

    I want to use direct mail feature of commerce server. But Direct mail is used basically for campaigning purpose. My requirement is to send an email to selected users after placing an order. (Order Confirmation Email/Shipping confirmation email). Can we achieve this using commerce server direct mail?

  16. MSDN Archive says:

    Direct Mail was not designed for this. Nothing is stoping you form doing this but I don’t see why you would use it for this purpose.

    -Max

  17. Vedavathi says:

    Hello Max,

    I need some information regarding the Commerce server pipelines. We are in the process of implementing pipelines in our project. We have started with Basket pipelines. We are just using the available pipelines with commerce server 2007. When I run the basket pipeline, I always get a warning message like, ‘Message: basketError: At least one item is out of stock’, although the item is in stock (Verified it by checking the InStockQuantity and Quantity of the Basket object). Can you please tell us what other properties this pipeline look for as it says the item is out of stock. Also we are running the pipeline after adding all the information to the basket like, lineitems, Shipping details, Payment details and just before executing the SaveAsOrder method. Is this the right place to call?

  18. MSDN Archive says:

    Turn on the pipeline log that sould give you more details.

    -Max

Comments are closed.

Skip to main content