WCF streaming inside data contracts

(A small break from the current WCF extensibility series, as I faced this issue helping a customer this week and I thought it would be worth sharing)

In order to support transferring of large messages (e.g., uploading or downloading a large file), WCF added support (since its first version) to streaming message transfer – unlike the default behavior, which is to buffer the entire message prior to sending it to the wire (or delivering it to the application layer, in the receiving end). Buffering has lots of advantages – it is faster, it allows for operations such as signing the body (which needs the whole content), reliable messaging (which may need to retransmit a message, and streams can often be read only once). However, there are situations, such as uploading a 1GB file to a server, when buffering the message would be too memory-intensive, if at all possible. For these cases streaming can be used.

Enabling streaming is simply a matter of setting the appropriate transfer mode in the binding. BasicHttpBinding, NetTcpBinding and NetNamedPipeBinding expose that property directly. For other binding types, you’ll need to convert them to a custom binding and set that property on the transport binding element itself. But to really get the benefits of streamed transfer, you need to use some data types which don’t need to buffer all the information prior to being serialized. Using a Message object directly (untyped message programming) certainly can be done, as you control the whole message layout, and it can be created based on classes which can emit the message parts on the fly (such as a XmlReader or a BodyWriter), but that’s too low-level for most applications (you need essentially to created the request / parse the response almost “from scratch” – just a little above dealing with raw bytes).

Another parameter type which is naturally “suited” for streaming scenarios are types which implement IXmlSerializable. The contract for such types is that they essentially control their whole serialization / deserialization. On serialization, the class WriteXml method is called, receiving a XmlWriter positioned at the wrapping element. At that point, the class can write as much information as needed, without needing to buffer anything in memory. On the ReadXml method, the class receives a XmlReader positioned again at the wrapping element, and the class can read as much information can consume it without having to buffer it all (but it should only read information pertaining to itself). Below is a simple example of an IXmlSerializable type which produces / consumes 10000 elements in the message, without having to buffer it.

  1. public class MyXmlSerializable : IXmlSerializable
  2. {
  3.     int total;
  4.     public XmlSchema GetSchema()
  5.     {
  6.         return null;
  7.     }
  8.  
  9.     public void ReadXml(XmlReader reader)
  10.     {
  11.         reader.ReadStartElement();
  12.         for (int i = 0; i < 10000; i++)
  13.         {
  14.             this.total += reader.ReadElementContentAsInt();
  15.         }
  16.  
  17.         reader.ReadEndElement();
  18.     }
  19.  
  20.     public void WriteXml(XmlWriter writer)
  21.     {
  22.         for (int i = 0; i < 10000; i++)
  23.         {
  24.             writer.WriteStartElement("item_" + i);
  25.             writer.WriteValue(i);
  26.             writer.WriteEndElement();
  27.         }
  28.     }
  29. }

IXmlSerializable types aren’t very friendly for simple operations, as the user still has to write code to handle all the serialization. For simple scenarios such as uploading / downloading files, for example, it would be cumbersome to have to write the code to read from the stream / write to the stream and convert it into XML. To make those scenarios easier to implement, WCF exposes on the service model a few capabilities to help with streaming. For operation or message contracts, if you define a single body parameter of type System.IO.Stream, WCF will map it to the whole message body, and the operation can be defined fairly simply, like in the example below. As far as a WCF operation is concerned, a Stream type is equivalent to a byte[] operation (they both map to the XML schema type xs:base64Binary directly). Notice that the type needs to be Stream, not any of its subclasses (MemoryStream, FileStream, etc). When writing the message to the wire, WCF will read the stream passed by the user, and write its bytes out. When reading the message, WCF will create its own read-only stream, and pass it to the user.

  1. [MessageContract]
  2. public class UploadFileRequest
  3. {
  4.     [MessageHeader]
  5.     public string fileName;
  6.     [MessageBodyMember]
  7.     public Stream fileContents;
  8. }
  9.  
  10. [ServiceContract]
  11. public interface IFileDownloader
  12. {
  13.     [OperationContract]
  14.     Stream DownloadFile(string fileName);
  15.     [OperationContract]
  16.     void UploadFile(UploadFileRequest request);
  17. }

Notice the restriction of one single body parameter, of type Stream – that’s when WCF will map the entire message body to the stream parameter. For most of the cases, this can be resolved by simply changing the operation signature and moving some parameters from the operation signature to the message header, which is the case of the UploadFile above – the operation would normally be defined as “void UploadFile(string fileName, Stream fileContents)”, but at that point both there are two parameters mapped to the body and the default (easy) mapping falls apart. But changing the operation to a MessageContract-based one is quite straightforward, and it solves most of the problems.

Interoperability issues

But the recommendation of changing the operation contract to make it WCF-happy many times doesn’t work with interoperability scenarios. In such cases, we want to consume an existing service provided by a 3rd party which we have no control over. At that point, we’re stuck with creating a message in the exact format as requested by the service. If the service expects the file contents (or any large binary data, for that matter) to be sent as part of a data contract graph, the proxy generated by svcutil / add service reference will contain such data contracts or types decorated with XML attributes (e.g., XmlType, XmlRoot, etc) and one of those types will contain a byte[] member. Byte arrays are buffered by nature (the array is stored in memory), and switching it to a Stream type won’t work – in the middle of the data contracts the control is handled to the serializers, and they don’t know how to handle Stream parameters like the WCF service model.

So we’re back to the no-Stream support case. Again, we can handcraft the message ourselves, but that usually requires a lot of effort (creating the XML “by hand”). The other alternative, IXmlSerializable is recognized by the serializers (both the DataContractSerializer, and the XmlSerializer, the two “default” serializers for WCF), so that’s where we can go. The main advantage of going the IXmlSerializable way is that we only need to change the class which contains the byte[] field, while the others (any wrapping types) can be left untouched. The task now is to implement the WriteXml and ReadXml methods in a way that they reflect the same contract as the “default” behavior for those classes.

To explain how to change one such class, I’ll define a simple service which contains a binary (byte[]) member nested in a data contract. I’ll use the XmlSerializer ([XmlSerializerFormat]) since that’s the serializer mostly used in interop scenarios, but the steps should be the same for DataContractSerializer types as well. Here’s the service:

  1. [XmlRoot(Namespace = "https://my.namespace.com/data")]
  2. public class RequestClass
  3. {
  4.     [XmlAttribute]
  5.     public string id;
  6.     [XmlElement]
  7.     public string token;
  8.     [XmlElement]
  9.     public Content content;
  10. }
  11.  
  12. [XmlType(Namespace = "https://my.namespace.com/data")]
  13. public class Content
  14. {
  15.     [XmlElement]
  16.     public string name;
  17.     [XmlElement]
  18.     public string extension;
  19.     [XmlElement]
  20.     public byte[] data;
  21. }
  22.  
  23. [ServiceContract(Namespace = "https://my.namespace.com/service")]
  24. [XmlSerializerFormat]
  25. public interface ISomeService
  26. {
  27.     [OperationContract]
  28.     void SendRequest(RequestClass request);
  29. }
  30.  
  31. public class SomeServiceImpl : ISomeService
  32. {
  33.     public void SendRequest(RequestClass request)
  34.     {
  35.         Console.WriteLine(
  36.             "Received request for {0}.{1}, with {2} bytes",
  37.             request.content.name,
  38.             request.content.extension,
  39.             request.content.data.Length);
  40.     }
  41. }

After running the service, I can point add service reference or svcutil to it to generate my proxy code:

svcutil https://localhost:8000/Service

This generates the following proxy code:

  1. //------------------------------------------------------------------------------
  2. // <auto-generated>
  3. // This code was generated by a tool.
  4. // Runtime Version:4.0.30319.1
  5. //
  6. // Changes to this file may cause incorrect behavior and will be lost if
  7. // the code is regenerated.
  8. // </auto-generated>
  9. //------------------------------------------------------------------------------
  10.  
  11.  
  12.  
  13. [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "4.0.0.0")]
  14. [System.ServiceModel.ServiceContractAttribute(Namespace="https://my.namespace.com/service", ConfigurationName="ISomeService")]
  15. public interface ISomeService
  16. {
  17.     
  18.     [System.ServiceModel.OperationContractAttribute(Action="https://my.namespace.com/service/ISomeService/SendRequest", ReplyAction="https://my.namespace.com/service/ISomeService/SendRequestResponse")]
  19.     [System.ServiceModel.XmlSerializerFormatAttribute()]
  20.     void SendRequest(RequestClass request);
  21. }
  22.  
  23. /// <remarks/>
  24. [System.CodeDom.Compiler.GeneratedCodeAttribute("svcutil", "4.0.30319.1")]
  25. [System.SerializableAttribute()]
  26. [System.Diagnostics.DebuggerStepThroughAttribute()]
  27. [System.ComponentModel.DesignerCategoryAttribute("code")]
  28. [System.Xml.Serialization.XmlTypeAttribute(Namespace="https://my.namespace.com/data")]
  29. public partial class RequestClass
  30. {
  31.     
  32.     private string tokenField;
  33.     
  34.     private Content contentField;
  35.     
  36.     private string idField;
  37.     
  38.     /// <remarks/>
  39.     [System.Xml.Serialization.XmlElementAttribute(Order=0)]
  40.     public string token
  41.     {
  42.         get
  43.         {
  44.             return this.tokenField;
  45.         }
  46.         set
  47.         {
  48.             this.tokenField = value;
  49.         }
  50.     }
  51.     
  52.     /// <remarks/>
  53.     [System.Xml.Serialization.XmlElementAttribute(Order=1)]
  54.     public Content content
  55.     {
  56.         get
  57.         {
  58.             return this.contentField;
  59.         }
  60.         set
  61.         {
  62.             this.contentField = value;
  63.         }
  64.     }
  65.     
  66.     /// <remarks/>
  67.     [System.Xml.Serialization.XmlAttributeAttribute()]
  68.     public string id
  69.     {
  70.         get
  71.         {
  72.             return this.idField;
  73.         }
  74.         set
  75.         {
  76.             this.idField = value;
  77.         }
  78.     }
  79. }
  80.  
  81. /// <remarks/>
  82. [System.CodeDom.Compiler.GeneratedCodeAttribute("svcutil", "4.0.30319.1")]
  83. [System.SerializableAttribute()]
  84. [System.Diagnostics.DebuggerStepThroughAttribute()]
  85. [System.ComponentModel.DesignerCategoryAttribute("code")]
  86. [System.Xml.Serialization.XmlTypeAttribute(Namespace="https://my.namespace.com/data")]
  87. public partial class Content
  88. {
  89.     
  90.     private string nameField;
  91.     
  92.     private string extensionField;
  93.     
  94.     private byte[] dataField;
  95.     
  96.     /// <remarks/>
  97.     [System.Xml.Serialization.XmlElementAttribute(Order=0)]
  98.     public string name
  99.     {
  100.         get
  101.         {
  102.             return this.nameField;
  103.         }
  104.         set
  105.         {
  106.             this.nameField = value;
  107.         }
  108.     }
  109.     
  110.     /// <remarks/>
  111.     [System.Xml.Serialization.XmlElementAttribute(Order=1)]
  112.     public string extension
  113.     {
  114.         get
  115.         {
  116.             return this.extensionField;
  117.         }
  118.         set
  119.         {
  120.             this.extensionField = value;
  121.         }
  122.     }
  123.     
  124.     /// <remarks/>
  125.     [System.Xml.Serialization.XmlElementAttribute(DataType="base64Binary", Order=2)]
  126.     public byte[] data
  127.     {
  128.         get
  129.         {
  130.             return this.dataField;
  131.         }
  132.         set
  133.         {
  134.             this.dataField = value;
  135.         }
  136.     }
  137. }
  138.  
  139. [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "4.0.0.0")]
  140. public interface ISomeServiceChannel : ISomeService, System.ServiceModel.IClientChannel
  141. {
  142. }
  143.  
  144. [System.Diagnostics.DebuggerStepThroughAttribute()]
  145. [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "4.0.0.0")]
  146. public partial class SomeServiceClient : System.ServiceModel.ClientBase<ISomeService>, ISomeService
  147. {
  148.     
  149.     public SomeServiceClient()
  150.     {
  151.     }
  152.     
  153.     public SomeServiceClient(string endpointConfigurationName) :
  154.             base(endpointConfigurationName)
  155.     {
  156.     }
  157.     
  158.     public SomeServiceClient(string endpointConfigurationName, string remoteAddress) :
  159.             base(endpointConfigurationName, remoteAddress)
  160.     {
  161.     }
  162.     
  163.     public SomeServiceClient(string endpointConfigurationName, System.ServiceModel.EndpointAddress remoteAddress) :
  164.             base(endpointConfigurationName, remoteAddress)
  165.     {
  166.     }
  167.     
  168.     public SomeServiceClient(System.ServiceModel.Channels.Binding binding, System.ServiceModel.EndpointAddress remoteAddress) :
  169.             base(binding, remoteAddress)
  170.     {
  171.     }
  172.     
  173.     public void SendRequest(RequestClass request)
  174.     {
  175.         base.Channel.SendRequest(request);
  176.     }
  177. }

Now, to the conversion. Unfortunately, there’s no automated way (AFAIK) to do the conversion, so we have to do it “by hand”. For me, the easiest way is to actually call the method once (using a small byte array value which shouldn’t have any memory issues) while monitoring the request on Fiddler (my favorite HTTP monitoring tool), and see what WCF actually wrote. By doing that we’ll see what the service expects:

  1. static void Main(string[] args)
  2. {
  3.     SomeServiceClient c = new SomeServiceClient();
  4.  
  5.     byte[] fileContents = new byte[10];
  6.     for (int i = 0; i < fileContents.Length; i++)
  7.     {
  8.         fileContents[i] = (byte)'a';
  9.     }
  10.  
  11.     RequestClass request = new RequestClass
  12.     {
  13.         id = "id",
  14.         token = "token",
  15.         content = new Content
  16.         {
  17.             name = "file",
  18.             extension = "ext",
  19.             data = fileContents,
  20.         },
  21.     };
  22.  
  23.     c.SendRequest(request);
  24. }

And that request shown in fiddler is the following:

  1. <s:Envelope xmlns:s="https://schemas.xmlsoap.org/soap/envelope/">
  2.     <s:Body xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">
  3.         <SendRequest xmlns="https://my.namespace.com/service">
  4.             <request id="id">
  5.                 <token xmlns="https://my.namespace.com/data">token</token>
  6.                 <content xmlns="https://my.namespace.com/data">
  7.                     <name>file</name>
  8.                     <extension>ext</extension>
  9.                     <data>YWFhYWFhYWFhYQ==</data>
  10.                 </content>
  11.             </request>
  12.         </SendRequest>
  13.     </s:Body>
  14. </s:Envelope>

Now that we know what the request looks like, we can start converting the Content class to IXmlSerializable. The generated proxy classes are marked as partial, which will actually help us isolate the IXmlSerializable code in a separate file (it’s a matter of personal preference, but I prefer to keep the generated proxy as unchanged as possible, in case I need to regenerate it). Let’s focus first on write (sending byte[] / Stream parameters to the server). Most data types are written to XmlWriter using its WriteValue method (it has many overloads), while binary data is written using the WriteBase64 method. Here’s a first attempt of implementing IXmlSerializable on that type:

  1. using System;
  2. using System.Xml;
  3. using System.Xml.Schema;
  4. using System.Xml.Serialization;
  5.  
  6. public partial class Content : IXmlSerializable
  7. {
  8.     public XmlSchema GetSchema()
  9.     {
  10.         return null;
  11.     }
  12.  
  13.     public void ReadXml(XmlReader reader)
  14.     {
  15.         throw new NotImplementedException();
  16.     }
  17.  
  18.     public void WriteXml(XmlWriter writer)
  19.     {
  20.         writer.WriteStartElement("name");
  21.         writer.WriteValue(this.name);
  22.         writer.WriteEndElement();
  23.  
  24.         writer.WriteStartElement("extension");
  25.         writer.WriteValue(this.extension);
  26.         writer.WriteEndElement();
  27.  
  28.         writer.WriteStartElement("data");
  29.         writer.WriteBase64(this.data, 0, this.data.Length);
  30.         writer.WriteEndElement();
  31.     }
  32. }

It builds fine. But when I try to run it, it fails with the exception below:

System.InvalidOperationException: There was an error reflecting type 'RequestClass'.
---> System.InvalidOperationException: There was an error reflecting property 'content'.
---> System.InvalidOperationException: There was an error reflecting type 'Content'.
---> System.InvalidOperationException: Only XmlRoot attribute may be specified for the type Content. Please use XmlSchemaProviderAttribute to specify schema type.

The problem is that the attribute [XmlType] applied to the type Content in the generated file is conflicting with the IXmlSerializable implementation. XmlType is used to control the XML schema of a type, and IXmlSerializable has its own way of providing schema. This can be fixed by commenting out the attribute on the generated file:

  1. /// <remarks/>
  2. [System.CodeDom.Compiler.GeneratedCodeAttribute("svcutil", "4.0.30319.1")]
  3. [System.SerializableAttribute()]
  4. [System.Diagnostics.DebuggerStepThroughAttribute()]
  5. [System.ComponentModel.DesignerCategoryAttribute("code")]
  6. //[System.Xml.Serialization.XmlTypeAttribute(Namespace="https://my.namespace.com/data")]
  7. public partial class Content
  8. {

Now it builds fine, it runs fine. Looking at the request in Fiddler, it is exactly the same as the previous one, so we’re in the right track. It’s possible that it had some differences (mostly likely due to things such as XML namespaces), in this case we’d have to go back and update the writing logic. We haven’t changed anything yet, but now we can move on to adding a stream parameter. First, let’s define a Stream property which will be used instead of the byte[] one:

  1. public partial class Content : IXmlSerializable
  2. {
  3.     private Stream stream;
  4.     public Stream DataStream
  5.     {
  6.         set { this.stream = value; }
  7.     }
  8.  
  9.     // rest of class ommitted
  10. }

Now we can change the WriteXml implementation. We can still use a small buffer to avoid writing bytes 1-1, but it will certainly be smaller than many MBs of large files.

  1. public void WriteXml(XmlWriter writer)
  2. {
  3.     writer.WriteStartElement("name");
  4.     writer.WriteValue(this.name);
  5.     writer.WriteEndElement();
  6.  
  7.     writer.WriteStartElement("extension");
  8.     writer.WriteValue(this.extension);
  9.     writer.WriteEndElement();
  10.  
  11.     writer.WriteStartElement("data");
  12.     byte[] buffer = new byte[1000];
  13.     int bytesRead;
  14.     do
  15.     {
  16.         bytesRead = this.stream.Read(buffer, 0, buffer.Length);
  17.         if (bytesRead > 0)
  18.         {
  19.             writer.WriteBase64(buffer, 0, bytesRead);
  20.         }
  21.     } while (bytesRead > 0);
  22.     writer.WriteEndElement();
  23. }

And the Write part is done.

MTOM considerations

Before we move on to the Read part, let me just spend some time discussing about MTOM. MTOM is a W3C recommendation which defines an optimization for transmitting binary data in SOAP messages. Typically, binary data is base64-encoded and written inline in the SOAP body. This is fine for many scenarios, but it has the drawback that base64-encoding increases the data size by ~33%. MTOM optimizes that by sending large binary data not inlined in the message, but as MIME attachments instead. They have a small overhead for some MIME headers, but the binary data is sent as-is, without any encoding, so the message doesn’t have the 33% size penalty.

In WCF, MTOM is implemented by an implementation of the XmlWriter abstract class (the internal class XmlMtomWriter). Whenever the user (or WCF code itself) writes binary data (WriteBase64), the MTOM writer will hold on to the data written and emit it as attachments after the whole message body has been written. By holding on to the data, the writer is essentially buffering it (since it needs to do it, as it needs to finish writing the message prior to writing the attachments), so the code above for WriteXml won’t do what we want in MTOM – it will still buffer the stream contents!

The solution for this problem is to hand over the Stream itself to the MTOM writer, and let it hold on to it until it’s ready to consume it. This is done via the IStreamProvider interface, and the XmlDictionaryWriter (a class derived from XmlWriter which is used internally in WCF) adds a new overload to WriteValue which takes an IStreamProvider parameter. In normal (i.e., non-MTOM) writers, this implementation simply copies the stream to the writer by calling WriteBase64 (as we did in our initial WriteXml implementation). The MTOM writer, however, holds on to the stream and will only consume it when it needs to.

This is the MTOM-aware version of the WriteXml method now:

  1. public void WriteXml(XmlWriter writer)
  2. {
  3.     writer.WriteStartElement("name");
  4.     writer.WriteValue(this.name);
  5.     writer.WriteEndElement();
  6.  
  7.     writer.WriteStartElement("extension");
  8.     writer.WriteValue(this.extension);
  9.     writer.WriteEndElement();
  10.  
  11.     writer.WriteStartElement("data");
  12.     XmlDictionaryWriter dictWriter = writer as XmlDictionaryWriter;
  13.     bool isMtom = dictWriter != null && dictWriter is IXmlMtomWriterInitializer;
  14.     if (isMtom)
  15.     {
  16.         dictWriter.WriteValue(new MyStreamProvider(this.stream));
  17.     }
  18.     else
  19.     {
  20.         // fall back to the original behavior
  21.         byte[] buffer = new byte[1000];
  22.         int bytesRead;
  23.         do
  24.         {
  25.             bytesRead = this.stream.Read(buffer, 0, buffer.Length);
  26.             if (bytesRead > 0)
  27.             {
  28.                 writer.WriteBase64(buffer, 0, bytesRead);
  29.             }
  30.         } while (bytesRead > 0);
  31.     }
  32.     writer.WriteEndElement();
  33. }
  34.  
  35. class MyStreamProvider : IStreamProvider
  36. {
  37.     Stream stream;
  38.     public MyStreamProvider(Stream stream)
  39.     {
  40.         this.stream = stream;
  41.     }
  42.  
  43.     public Stream GetStream()
  44.     {
  45.         return this.stream;
  46.     }
  47.  
  48.     public void ReleaseStream(Stream stream)
  49.     {
  50.     }
  51. }

And now the Write part is really done.

Reading

In order to test the reading part I’ll update the test server with an extra operation which returns a byte[] field nested inside a data contract graph.

  1. [XmlRoot(Namespace = "https://my.namespace.com/data")]
  2. public class ResponseClass
  3. {
  4.     [XmlAttribute]
  5.     public string id;
  6.     [XmlElement]
  7.     public string token;
  8.     [XmlElement]
  9.     public Content content;
  10. }
  11.  
  12. [ServiceContract(Namespace = "https://my.namespace.com/service")]
  13. [XmlSerializerFormat]
  14. public interface ISomeService
  15. {
  16.     [OperationContract]
  17.     void SendRequest(RequestClass request);
  18.     [OperationContract]
  19.     ResponseClass GetResponse(int dataSize);
  20. }
  21.  
  22. public class SomeServiceImpl : ISomeService
  23. {
  24.     public void SendRequest(RequestClass request)
  25.     {
  26.         Console.WriteLine(
  27.             "Received request for {0}.{1}, with {2} bytes",
  28.             request.content.name,
  29.             request.content.extension,
  30.             request.content.data.Length);
  31.     }
  32.     public ResponseClass GetResponse(int dataSize)
  33.     {
  34.         byte[] data = new byte[dataSize];
  35.         for (int i = 0; i < dataSize; i++) data[i] = (byte)'b';
  36.         return new ResponseClass
  37.         {
  38.             id = "resp",
  39.             token = "tkn",
  40.             content = new Content
  41.             {
  42.                 name = "resp",
  43.                 extension = "txt",
  44.                 data = data,
  45.             },
  46.         };
  47.     }
  48. }

Updating the client to use it: re-generated the proxy with svcutil/ASR, comment out the [XmlType] attribute on the Content class, and update the Main method:

  1. static void Main(string[] args)
  2. {
  3.     SomeServiceClient c = new SomeServiceClient();
  4.     c.GetResponse(20);
  5. }

And it fails when run:

System.ServiceModel.CommunicationException: Error in deserializing body of reply message for operation 'GetResponse'.
---> System.InvalidOperationException: There is an error in XML document (1, 363).
---> System.NotImplementedException: The method or operation is not implemented.
at Content.ReadXml(XmlReader reader)

Now we need to implement ReadXml. The idea is similar to the WriteXml: look at the response on fiddler

  1. <s:Envelope xmlns:s="https://schemas.xmlsoap.org/soap/envelope/">
  2.     <s:Body xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">
  3.         <GetResponseResponse xmlns="https://my.namespace.com/service">
  4.             <GetResponseResult id="resp">
  5.                 <token xmlns="https://my.namespace.com/data">tkn</token>
  6.                 <content xmlns="https://my.namespace.com/data">
  7.                     <name>resp</name>
  8.                     <extension>txt</extension>
  9.                     <data>YmJiYmJiYmJiYmJiYmJiYmJiYmI=</data>
  10.                 </content>
  11.             </GetResponseResult>
  12.         </GetResponseResponse>
  13.     </s:Body>
  14. </s:Envelope>

And start reading from the XmlReader into the class. Notice that the reader is positioned at the wrapping element (<content>), so to access the fields we need first to read past it. After that we can start reading the data fields themselves:

  1. public void ReadXml(XmlReader reader)
  2. {
  3.     reader.ReadStartElement(); //wrapping element
  4.  
  5.     this.name = reader.ReadElementContentAsString();
  6.     this.extension = reader.ReadElementContentAsString();
  7.  
  8.     MemoryStream ms = new MemoryStream();
  9.     byte[] buffer = new byte[1000];
  10.     int bytesRead;
  11.     reader.ReadStartElement();
  12.     do
  13.     {
  14.         bytesRead = reader.ReadContentAsBase64(buffer, 0, buffer.Length);
  15.         ms.Write(buffer, 0, bytesRead);
  16.     } while (bytesRead > 0);
  17.  
  18.     this.data = ms.ToArray();
  19.     reader.ReadEndElement();
  20.  
  21.     reader.ReadEndElement(); //wrapping element
  22. }

This works fine, but at that point we’re still buffering the response. Unfortunately we can’t avoid it – since it’s in the middle of the body (not the whole body), WCF needs to read all of the content data to be able to finish reading the rest of the message. If this is a problem, however, we can shift the buffering from memory to another location – such as disk – which can hold on to very large structures a lot easier. Here’s the modified code. Notice that I didn’t expose the file stream as a property, but as a method instead, to make it clearer that it’s not just retrieving the value of a field, but it’s doing something else instead.

  1. public partial class Content : IXmlSerializable
  2. {
  3.     private Stream stream;
  4.     private string streamFileName;
  5.  
  6.     public Stream DataStream
  7.     {
  8.         set { this.stream = value; }
  9.     }
  10.  
  11.     public Stream GetDataStream()
  12.     {
  13.         return File.OpenRead(this.streamFileName);
  14.     }
  15.  
  16.     public void WriteXml(XmlWriter writer)
  17.     {
  18.         writer.WriteStartElement("name");
  19.         writer.WriteValue(this.name);
  20.         writer.WriteEndElement();
  21.  
  22.         writer.WriteStartElement("extension");
  23.         writer.WriteValue(this.extension);
  24.         writer.WriteEndElement();
  25.  
  26.         writer.WriteStartElement("data");
  27.         XmlDictionaryWriter dictWriter = writer as XmlDictionaryWriter;
  28.         bool isMtom = dictWriter != null && dictWriter is IXmlMtomWriterInitializer;
  29.         if (isMtom)
  30.         {
  31.             dictWriter.WriteValue(new MyStreamProvider(this.stream));
  32.         }
  33.         else
  34.         {
  35.             // fall back to the original behavior
  36.             byte[] buffer = new byte[1000];
  37.             int bytesRead;
  38.             do
  39.             {
  40.                 bytesRead = this.stream.Read(buffer, 0, buffer.Length);
  41.                 if (bytesRead > 0)
  42.                 {
  43.                     writer.WriteBase64(buffer, 0, bytesRead);
  44.                 }
  45.             } while (bytesRead > 0);
  46.         }
  47.         writer.WriteEndElement();
  48.     }
  49.  
  50.     public void ReadXml(XmlReader reader)
  51.     {
  52.         reader.ReadStartElement(); //wrapping element
  53.  
  54.         this.name = reader.ReadElementContentAsString();
  55.         this.extension = reader.ReadElementContentAsString();
  56.  
  57.         string tempFileName = Path.GetTempFileName();
  58.         using (FileStream fs = File.Create(tempFileName))
  59.         {
  60.             byte[] buffer = new byte[1000];
  61.             int bytesRead;
  62.             reader.ReadStartElement();
  63.             do
  64.             {
  65.                 bytesRead = reader.ReadContentAsBase64(buffer, 0, buffer.Length);
  66.                 fs.Write(buffer, 0, bytesRead);
  67.             } while (bytesRead > 0);
  68.  
  69.             reader.ReadEndElement();
  70.         }
  71.  
  72.         reader.ReadEndElement(); //wrapping element
  73.     }
  74.  
  75.     public XmlSchema GetSchema()
  76.     {
  77.         return null;
  78.     }
  79.  
  80.     class MyStreamProvider : IStreamProvider
  81.     {
  82.         Stream stream;
  83.         public MyStreamProvider(Stream stream)
  84.         {
  85.             this.stream = stream;
  86.         }
  87.  
  88.         public Stream GetStream()
  89.         {
  90.             return this.stream;
  91.         }
  92.  
  93.         public void ReleaseStream(Stream stream)
  94.         {
  95.         }
  96.     }
  97. }

Hope this helps!

[Code for this post]