Custom XML Validator for Throwing Multiple Errors
When applications / systems integrate with BizTalk via published WCF-based web services I often promote the idea that messages received by this web service should be validated before the sender of the message receives an acknowledgement that message was successfully delivered. Where a message fails validation I would also like to be able to return the reasons why validation failed. Implementing validation is fairly simple within BizTalk, but returning all the reasons why validation failed is slightly more complex.
Standard Approaches to Validation
The default XMLReceive pipeline uses the standard XML Disassembler component to identify the deployed schema against which the message should be validated and, optionally, will validate the received XML message against this identified schema. When configuring this validation you will need to set the “ValidateDocument” property to True, and provide the fully qualified assembly names for the schemas you want to validate.
Once configured in this way the incoming messages are matched against the schemas in the DocumentSpecNames property, and then validated against the schema. As soon as a validation error is encountered, the XML Validator throws an exception with the inner exception containing the reason for the validation failure. The resulting error would look something like this, indicating that the MonthNumber element probably has alphanumeric text as content, but the MonthNumber element is defined as xs:int:
There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "Pipeline " Receive Port: "Test Services" URI: "/TestServices/Test.svc" Reason: The document failed to validate because of the following error:"The 'https://test.org/schemas/1.3.0:MonthNumber' element has an invalid value according to its data type." .
The downside of implementing this method of validation is that you have to essentially hard-code the schema names in the configuration for the pipeline.
The second approach is to create a custom pipeline, composed of the standard XML Disassembler and standard XML Validator components. This essentially moves the validation logic from the XML Disassembler to the XML Validator component. A major benefit of using this approach is that you do not need to specify the document schema to validate the message against … the pipeline component will use the message's target namespace and root element name information to find the appropriate schema. The result is that the pipeline will dynamically be able to identify the schema for the received message, and then dynamically validated the message against the identified schema, making the pipeline a generic pipeline that can be used to validate any incoming message. As with the first approach, however, this approach will also only raise the first validation error.
Before discussing how we can identify multiple validation errors, let’s discuss how we can return error information for an invalid message to the sender of the message.
Dealing with Invalid Messages
When an incoming message fails validation it can be regarded as a “poison message” (https://winterdom.com/2007/02/poisonmessagesandorderedprocessing), and should not be allowed to process further through the solution. Instead, details of why the message failed validation should be returned to the sender so that they can make the necessary changes in their solution to correct the cause of the failure.
Using the WCF-WSHttp and WCF-BasicHttp adapters achieving this is rather easy. By checking the “Include exception detail in faults” flag in the “Messages” tab for the WCF settings, BizTalk will return the exception from the pipeline to the WCF-WSHttp or WCF-BasicHttp adapter, which will then return the exception as a SOAP fault to the sender.
The result will be that when the sender application submits the message to a BizTalk web service, the application would not receive the HTTP 200 acknowledgement until the message has validated successfully. Where the message fails validation, the sender will receive the validation error as part of a SOAP Fault, returned with an HTTP 500 error.
The sender system would then need to correct the message and resubmit the message to the BizTalk web service. If the message fails validation a number of times, the process of sending, receiving a single error, and resubmitting the message will be a very time-consuming and frustrating process.
(Of course, the best would be for the sender to be validating the message against the schema BEFORE they send it, but I have seen it happen far too often where this is not done.)
Solution
The ideal solution to this would be to have the validation process identify as many failures as possible and return a message to the sender with the detail for all the failures.
Saravana Kumar developed a pipeline component that addresses this requirement on the basis that all the validation errors are identified and then used to formulate a new message. This new message is then returned as a standardised Error XML message to the sender (https://blogs.digitaldeposit.net/saravana/post/2008/04/05/Extended-XmlValidation-Pipeline-Component.aspx).
While this component performs the core functionality of identifying all the validation errors, I wanted something a little more direct. Imagine a scenario where you have an event-based message exchange hub, using one-way WCF-WSHttp receive locations for receiving messages of varying structural complexity from partners via SOAP-based communication. A key principle of this architecture would be that the sender of a message should receive a communication acknowledgement (HTTP 200) ONLY if the message has been validated successfully. Therefore, if there are any errors encountered BEFORE the message has been received by the BizTalk Message Box, these errors are to be returned to the sender as a SOAP fault, rather than as a new message. This would result in the sender receiving an HTTP 200 acknowledgement if the message was successfully validated, or an HTTP 500 error, and the SOAP fault with all validation errors, if the validation failed.
Based on these requirements I developed a new custom XML Validator pipeline component. This component also needed a configurable maximum error limit, so that the validation process would halt if a specified number of errors in the validation process were encountered. In a custom receive pipeline the standard XML Disassembler component will still be used to identify the appropriate message schema, but then this custom XML Validator component will then validate the message against the message’s schema, gather all the validation errors (up to the maximum error count), and raise a new exception with detail for each validation error included. This exception would then be returned to the sender of the message in the same way as described earlier in this blog.
The result is that when a message submitted fails validation, the validation errors are returned to the sender as a SOAP fault, as shown below:
<s:Envelope xmlns:s="https://www.w3.org/2003/05/soap-envelope" xmlns:a="https://www.w3.org/2005/08/addressing">
<s:Header>
<a:Action s:mustUnderstand="1">https://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher/fault</a:Action>
</s:Header>
<s:Body>
<s:Fault>
<s:Code>
<s:Value>s:Receiver</s:Value>
<s:Subcode>
<s:Value xmlns:a="https://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher">a:InternalServiceFault</s:Value>
</s:Subcode>
</s:Code>
<s:Reason>
<s:Text xml:lang="en-ZA">There was a failure executing the receive pipeline: "BizTalk.Common.Pipelines.XMLValidatingReceive, BizTalk.Common.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0ba159bca7ecb7f3" Source: "CustomXMLValidator Component" Receive Port: "Test Services" URI: "/TestServices/Test.svc" Reason: XML validation failed for a "https://ijs.gov.za/schemas/sajsvc/messages/DocketReadyNotification/3.0#DocketReadyNotification" message, with a message ID of "0d723957-861b-4896-9b3d-9985daa16fe2". Error Message(s):
Line: 28, Position: 38, Error: The 'https://test.org/schemas/1.3.0:MonthNumber' element has an invalid value according to its data type.
Line: 29, Position: 37, Error: The 'https://test.org/schemas/1.3.0:YearNumber' element has an invalid value according to its data type.
Line: 40, Position: 13, Error: The element 'DocumentCategoryID' in namespace 'https://test.org/schemas/1.3.0' has incomplete content. List of possible elements expected: 'https://test.org/schemas/1.3.0:ID'.
Line: 67, Position: 12, Error: The element 'CaseCharge' in namespace 'https://test.org/schemas/3.0' has incomplete content. List of possible elements expected: 'https://test.org/schemas/1.3.0:ChargeStatute'.</s:Text>
</s:Reason>
<s:Detail>
<ExceptionDetail xmlns="https://schemas.datacontract.org/2004/07/System.ServiceModel" xmlns:i="https://www.w3.org/2001/XMLSchema-instance">
<HelpLink i:nil="true"/>
<InnerException i:nil="true"/>
<Message>There was a failure executing the receive pipeline: "BizTalk.Common.Pipelines.XMLValidatingReceive, BizTalk.Common.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0ba159bca7ecb7f3" Source: "CustomXMLValidator Component" Receive Port: "Test Services" URI: "/TestServices/Test.svc" Reason: XML validation failed for a "https://ijs.gov.za/schemas/sajsvc/messages/DocketReadyNotification/3.0#DocketReadyNotification" message, with a message ID of "0d723957-861b-4896-9b3d-9985daa16fe2". Error Message(s):
Line: 28, Position: 38, Error: The 'https://test.org/schemas/1.3.0:MonthNumber' element has an invalid value according to its data type.
Line: 29, Position: 37, Error: The 'https://test.org/schemas/1.3.0:YearNumber' element has an invalid value according to its data type.
Line: 40, Position: 13, Error: The element 'DocumentCategoryID' in namespace 'https://test.org/schemas/1.3.0' has incomplete content. List of possible elements expected: 'https://test.org/schemas/1.3.0:ID'.
Line: 67, Position: 12, Error: The element 'CaseCharge' in namespace 'https://test.org/schemas/3.0' has incomplete content. List of possible elements expected: 'https://test.org/schemas/1.3.0:ChargeStatute'.</Message>
<StackTrace>at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkServiceInstance.EndOperation(IAsyncResult result)
at AsyncInvokeEndEndTwoWayMethod(Object , Object[] , IAsyncResult )
at System.ServiceModel.Dispatcher.AsyncMethodInvoker.InvokeEnd(Object instance, Object[]& outputs, IAsyncResult result)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeEnd(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage7(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</StackTrace>
<Type>Microsoft.BizTalk.Message.Interop.BTSException</Type>
</ExceptionDetail>
</s:Detail>
</s:Fault>
</s:Body>
</s:Envelope>
Formatting of this SOAP fault will be a challenge for another day … anyone have ideas on this?
The source code for this CustomXMLValidator component is available from my SkyDrive, correction my OneDrive.