Building a Secure Composite Duplex

I'm getting this error message even though I have security enabled for my service:

Unhandled Exception: System.InvalidOperationException: The response message must be protected. This is required by an operation of the contract ('ICalculator', 'Microsoft.ServiceModel.Samples'). The protection must be provided by the binding ('CustomBinding', 'tempuri.org/').

This is sometimes a symptom of incorrect layering between security and composite duplex. Composite duplex correlates two channels together. If the security binding element is below composite duplex in the channel stack, then you'll get this error message because the security channel that is protecting the requests is not able to protect the responses that come in along the back channel.

The solution is to reverse the ordering and put security on top of the composite duplex channel. Additionally, you need to specify the use of SecureConversation for that security channel. The security method that you are actually using (user name/password, certificates, Kerberos, whatever) should be specified as the bootstrapping method for the secure conversation. Here's an example of a server with the right layering.

 CustomBinding binding = new CustomBinding();
SecurityBindingElement security = SecurityBindingElement.CreateAnonymousForCertificateBindingElement();
binding.Elements.Add(SecurityBindingElement.CreateSecureConversationBindingElement(security));
binding.Elements.Add(new CompositeDuplexBindingElement());
binding.Elements.Add(new OneWayBindingElement());
binding.Elements.Add(new TextMessageEncodingBindingElement());
binding.Elements.Add(new HttpTransportBindingElement());
ServiceHost host = new ServiceHost(typeof(CalculatorService), new Uri("localhost:8000/"));
host.Credentials.ServiceCertificate.SetCertificate(StoreLocation.CurrentUser, StoreName.My, X509FindType.FindBySubjectName, "localhost");
host.AddServiceEndpoint(typeof(ICalculator), binding, "");
host.Open();
Console.WriteLine("Press <ENTER> to terminate service.");
Console.ReadLine();
host.Close();

Next time: Getting the Client's Password