Testing in a SOA world

SOA brings change.  It is a change to the way we do IT business.  No question of it. Anyone who has tried to 'tack' SOA onto the side of an organization has seen the resistence that this generates.  "We've always done it that way before... why change?" 

One place where SOA has an impact, but few people speak openly about it, is the change that SOA has on the world of software testing.

There are some huge changes here:

1) Regression testing: The only hope you have for insuring that your service can handle itself in a changing environment is to create automated regression tests.  This was optional in the past. Now it is both Required and quite Feasable.  Since services have no User Interface, there is no need to worry about whether the control has moved. Maintaining an automated regression test script is much easier.

2) Boundary testing: If the intended use of a service is a good thing, what about the unintended use of a service?  Can the service survive being hit with FAR more requests than it was designed for?  Does it throttle itself?  Does it protect itself?  Can private data leak out? 

3) Integration testing: A LOT of the capability of a Service Oriented business app will move to the composition layer.  Testing the services helps to establish a baseline to show that the defects should be avoidable below the service line, but many of the bugs will occur because a composed service assumed that an underlying service would have a side effect that it does not have, (or vice versa).  As defects are found in Integration testing, the test cases need to be updated at the service level to insure that assumptions are constrained and tested.

4) Stub testing: you may need to test the composition layer before a service is available or while it is only 'local' instead of 'enterprise'.  For that reason, the test team needs to be able to generate 'stub services' that apps can call that behave in a manner that is compatible with the service definition, but has far less effect.  Otherwise, integrated testing is a joke.

5) New service validation and service compatibility validation: if a new service is entering an environment, and the goal is for the users of the existing service to transition over to it, then there has to be a way to test the new service to insure that it is compatible with the existing one.  Automated regression tests that were designed to test the existing service need to be pointed to the new service and failures noted.  Note that the team developing the new service is not likely to be the same team as the one that developed the prior one, so source code for the regression test must be available and shared and documented and stable.  This requires a level of 'test team integration' that many organizations will find challenging.

This is, I'm sure, a subset of the changes that SOA brings to the world of testing.  I encourage those who are involved in testing to share other ideas and concerns that they have come across with respect to SOA development.