Test Framework for Rapid Test Case Development

Article
09/18/2004

Firstly apologies for the lack of activity on my blog, I have several reasonable sized pieces of work that I’ve been trying to finish off ready for posting, this is the one of those pieces, and I wanted to get this one out now as I’m off to Redmond for a couple of weeks and if I don’t do it now, it ain’t gonna happen for a another month!

So, we all understand the value and importance of testing, but, there is of course a cost in terms of developing the volume of test cases in order to get sufficient test coverage for a given Biztalk solution or any other solution for that matter. There are of course many aspects to testing enterprise systems including unit tests, functional tests, integration tests, performance tests, user acceptance tests and stress tests. Functional and unit tests typically benefit from being automated; this helps to provide a metric to measure progress during the development process as well as helping to prevent regressions. Of course, after a solution has gone live, regression testing should be performed on any patches that are applied to the solution, so again, a suite of automated tests can give a high degree of confidence that a regression will not take place.

Recently, I started to use NUnit, I have to say I really like NUnit, but it didn’t quite meet my needs for testing end to end Biztalk scenarios, what was needed was a frame work to enable test cases to be created very quickly with little code, I also wanted to be able to change Url’s for receive and send ports etc without recompiling my test code so that I could use the same test cases for my development and production environments.

Enter the “Test Framework for Rapid Test Case Development”! (I really need to think if a decent name for it J). The approach that the Framework takes is very simple, firstly it treats Biztalk as a black box for the most part, it uses the notion that test cases are composed from many discrete test steps which maybe composed to form a single test case. A test case has three phases, setup, execution and tear down, each of these three phases may be comprised of multiple test steps. Further, a test step may use a validation step, the purpose of which is to validate that certain conditions are true, for example a stream containing Xml may be validated against a schema. The cleanup phase is always executed for each test that is started, the approach being that each test should leave the system in the same state that it found it as far as possible.

The individual test steps are .Net classes that are re-usable across different tests and even within the same test, the Framework creates them by inspecting the type name and assembly path specified in the Xml for that step. The Xml configuration is passed to the test step, the format of the configuration for each step is entirely up to the test step, so only the test step needs to understand its configuration. The interface that test steps must implement is shown below along with the Xml for the HTTP Post step:

public interface ITestStep

{

void Execute(XmlNode testConfig);

}

<SourcePath>.\TestData\InDoc1.xml</SourcePath>

<DestinationUrl>https://localhost/TestFrameworkDemo/BTSHTTPReceive.dll</DestinationUrl>

</TestStep>

The advantage of this approach is that test cases can be developed very quickly by simply authoring an Xml configuration file, the Xml could even be generated. Once the Xml is created, it is driven by constructing the TestDriver and then calling Run, the following code snippet illustrates how it can be run from NUnit:

[Test]

public void Http_To_File_Test_01()

{

TestExecuter testDriver = new TestExecuter(@".\TestCases\Http_To_File_Test_01.xml");

testDriver.RunTest();

}

Let’s look at a test case for a Biztalk scenario. An Xml File is submitted to a one way HTTP receive location, the receive port is bound to an orchestration that is activated and then waits for a second message that is received on a FILE receive location, at which point the orchestration transmits a message to a FILE send port. This scenario could be created by using an HTTP post step, a FILE creation step, and a FILE validation step which reads the FILE written to disc validates it against a specific schema and performs a number of XPath queries to test key fields in the data are correct.

The Test Framework is driven from an Xml configuration file, the Xml for the test case described above can be seen below:

</TestSetup>

<SourcePath>.\TestData\InDoc1.xml</SourcePath>

<DestinationUrl>https://localhost/TestFrameworkDemo/BTSHTTPReceive.dll</DestinationUrl>

</TestStep>

<SourcePath>.\TestData\InDoc1.xml</SourcePath>

<CreationPath>.\Rec_01\InDoc1.xml</CreationPath>

</TestStep>

<FilePath>.\Rec_02\InDoc1.xml</FilePath>

<XmlSchemaPath>.\TestData\PurchaseOrder.xsd</XmlSchemaPath>

<XmlSchemaNameSpace>https://SendMail.PurchaseOrder</XmlSchemaNameSpace>

<XPathValidation query="/*[local-name()='PurchaseOrder' and namespace-uri()='https://SendMail.PurchaseOrder']/*[local-name()='PONumber' and namespace-uri()='']">PONumber_0</XPathValidation>

</XPathList>

</ValidationStep>

</TestStep>

</TestExecution>

<FileToDeletePath>.\Rec_02\InDoc1.xml</FileToDeletePath>

<FileToDeletePath>.\Rec_01\InDoc1.xml</FileToDeletePath>

</TestStep>

</TestCleanup>

</TestCase>

The output from the test case is written to the NUnit console window and is shown below:

-------------------------------------------------------------------------------

S T A R T

Test: Http_To_File_Test_01 started @ 15:50:18.097 18/09/2004

-------------------------------------------------------------------------------

Setup Test: Http_To_File_Test_01

Execute Test: Http_To_File_Test_01

Step: Microsoft.Services.UK.TestFramework.HttpPostStep started @ 15:50:18.097 18/09/2004

Info: HttpRequestResponseStep about to post data from File: .\TestData\InDoc1.xml to the Url: https://localhost/TestFrameworkDemo/BTSHTTPReceive.dll

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Data: HttpPostStep response data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Step: Microsoft.Services.UK.TestFramework.HttpPostStep ended @ 15:50:19.469 18/09/2004

Step: Microsoft.Services.UK.TestFramework.FileCreateStep started @ 15:50:19.469 18/09/2004

Info: FileCreateStep about to copy the data from File: .\TestData\InDoc1.xml to the File: .\Rec_01\InDoc1.xml

Step: Microsoft.Services.UK.TestFramework.FileCreateStep ended @ 15:50:19.469 18/09/2004

Step: Microsoft.Services.UK.TestFramework.FileValidateStep started @ 15:50:19.469 18/09/2004

Info: FileXmlValidateStep validating file: .\Rec_02\InDoc1.xml

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Data: File data to be validated

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

<?xml version="1.0" encoding="utf-8"?><ns0:PurchaseOrder xmlns:ns0="https://SendMail.PurchaseOrder">

<PONumber>PONumber_0</PONumber>

<Email>Email_0</Email>

</CustomerInfo>

<Count>Count_0</Count>

</Description>

</ns0:PurchaseOrder>

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Validation: Microsoft.Services.UK.TestFramework.XmlValidationStep started @ 15:50:19.479 18/09/2004

Info: XmlValidationStep evaluting XPath /*[local-name()='PurchaseOrder' and namespace-uri()='https://SendMail.PurchaseOrder']/*[local-name()='PONumber' and namespace-uri()=''] equals "PONumber_0"

Validation: Microsoft.Services.UK.TestFramework.XmlValidationStep ended @ 15:50:19.479 18/09/2004

Step: Microsoft.Services.UK.TestFramework.FileValidateStep ended @ 15:50:19.479 18/09/2004

Tear Down Test: Http_To_File_Test_01

-------------------------------------------------------------------------------

Test: Http_To_File_Test_01 ended @ 15:50:19.479 18/09/2004

P A S S

-------------------------------------------------------------------------------

I’ve created a workspace on GotDotNet to host the Test Framework, the first version has the following test steps:

ExecuteCommandStep

FileCreateStep

FileDeleteStep

FileValidateStep

HttpPostStep

HttpRequestResponseStep

MSMQQueuePurgeStep

MSMQReadStep

MSMQWriteStep

And the following validation steps:

BinaryValidation

XmlValidationStep

I’ve used this version of the Framework to test some pretty complex Biztalk scenarios, but there are still plenty of other test steps that would be useful. I’ve included some very good ideas into the Framework from my colleague Greg Beech who has been testing Biztalk enterprise solutions in the Microsoft Solution Development Center for the last three years, when we compared notes we found out that we had similar approaches; one approach tied the steps together using code the other using Xml.

What would be really very cool is if we can get some community collaboration behind this, if other people find the framework useful and contribute other test and validation steps, I will merge them into the Framework so that every one benefits from them. The Readme.htm in the work space contains more detailed information on the Framework.

Test Framework for Rapid Test Case Development

Additional resources