In term of testing, here is some practice:
· Testing your service in production. In many case, it is easy to deploy your service to Windows Azure, and not expose to end user. This enable you can do end to end testing as early as possible. Also, you don’t need spend time to simulate your product environment any more. And you can find issues only happens in production.
· Deployment your service to PPE (Pre Production Environment), and allow internal customer to use your service and test your service.
· Design a ping protocol for your service (for example, for a web site, it is a HTTP URL), and use external provider, Gomez to monitoring your service.
· Implement external Application Monitoring. Unless the ping protocol, which only make sure your service is on or not, but not whether it functional well. The application monitoring will run common user scenario tests in production as real user, and monitoring the availability of the functionality of your service.
· not only You need test the functionality of your service, you also need to test other part of the service, such as logging, scalability , and error handle and fault tolerant to other dependent services,etc
· Implement continuous integration and continuous deployment. Continuous integration is a software development practice where engineers integrate frequently, leading to multiple integrations per day. Each integration is verified by an automated build and test to detect integration errors as quickly as possible. In your service case, you can implement a continuous deployment process, such as build your service, deployment it to test environment, and run sign-off tests against the test service. The process should be automatically happened. Enable this for your service enable you fast release your service, such as RTO-1.
· Experimental in production. It is possible that you can use your real user to test your service, such as some new idea or new implementation. You need to implement an risk and exposition control. For example, it is possible that you deploy your beta service to your stage environment, but both product and staging get request from the same request. You can config the stage deployment to take 1% of the request (such as adjust polling request frequency), so that it wouldn’t have big impact on many users. Note, it use the same ways as ramp-up deployment, which deploy your service in an incremental way.
· Use a well-know test framework, such as MsTest. The reason is that you can take advantages of other people's work related to mstest, such as running form the cloud, loading testing ,etc.