DataContractJsonSerializer and the “unexpected namespace” error
I’ve seen this error pop up in the forum enough times that I think it’s worth a blog post. JSON (www.json.org) is a very simple data format, and, unlike XML, doesn’t have the concept of namespaces – all it knows about are primitive types (strings, booleans, numbers), sequences (arrays) and records (objects). So it’s strange when out of the blue the DataContractJsonSerializer throws an exception saying that it encountered an unexpected namespace. This post explains the error, and shows a workaround for that. If you don’t care about the details, simply jump to the last section and you should get unblocked.
In the beginning, everything was XML
When WCF was first created (along with .NET Framework 3.0), it was designed as a way to create and consume XML-based web services. Many of the building blocks of the platform reflect that original assumption: the Message type is essentially XML based, the serializers can write objects to XmlWriter instances (and read objects from XmlReader instances). Even the type definitions of data contracts (and service and operation contracts) have XML concepts, such as namespaces. Here’s a typical definition of a data contract:
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyType
{
[DataMember]
public string str;
}
Enter JSON and the Web
On its first major upgrade (.NET Framework 3.5), WCF received some new bindings / serializers, to work with web-based clients (mostly AJAX-based) which were getting more and more prevalent. JSON was the data format used by most all web clients, so it was added to the framework on 3.5. The idea was that the same contracts (data / operation) which could be used to expose XML web services could also be reused for exposing the same services to the AJAX/JSON world as well.
Since everything in WCF was XML-based, the team decided to implement a XML interface to talk to JSON. The idea was to reuse all the WCF infrastructure, but instead of reading/writing angle brackets in the wire (a.k.a. “normal” XML), WCF would simply write JSON. And with that came to life the JsonReaderWriterFactory, which could create and XML reader/writer which could consume/produce JSON, according to the mapping described at https://msdn.microsoft.com/en-us/library/bb924435.aspx.
But JSON doesn’t have the concept of namespaces, and since we’re now writing JSON using an XmlWriter interface (with methods such as WriteStartElement(string prefix, string localName, string namespaceUri)), the XML/JSON writer would throw if one tried to write an element whose namespace was not empty. That would prevent a silent data loss scenario, where the user thinks the writer is taking the value it’s passing and doing something with it, and later when it tries to read the information wouldn’t be there. It’s better to simply be upfront about what is not supported and throw right away.
We now had a way to plug JSON into the WCF stack – it could be read/written as XML. But what about the serialization of types, which by default contain namespaces (if one is not specified in the [DataContract] attribute, the CLR namespace is used to create one). We could pass a XML/JSON writer to the DataContractSerializer, but when it tried to write the type name (along with its namespace), the writer would simply throw. So a new serializer was added to WCF, the DataContractJsonSerializer (DCJS), which knew how to ignore the namespaces of the types when writing it into “XML”. For example, in the type MyType above, the DCJS would treat as if it were declared as [DataContract(Name = “MyType”, Namespace = “”)]. So everything at that point everything was fine – the JSON serializer would never write namespaces, so the JSON writer would never complain.
Extension data and the exception
But if the JSON serializer never writes namespaces, why does this exception shows up every once in a while? The problem is with extension data – the IExtensibleDataObject interface (https://msdn.microsoft.com/en-us/library/system.runtime.serialization.iextensibledataobject.aspx). The main idea behind that interface is that by implementing it, when an instance of that type is being deserialized, it would store all data that it doesn’t know of, and when the instance is being serialized, it would write those back exactly as they were originally. This supports, for example, round-trip of data contracts between different versions of the same contract.
The problem starts with the exactly above – that’s the contract of IExtensionDataObject. An object which implements IEDO is claiming that it can receive data that it doesn’t know, and it will write it out in a way that whoever produced that data will consume and understand it. So when implementing the DCJS the WCF team had a decision to make: break the IEDO contract (by ignoring the namespaces there), or simply throw at that point. Following the same decision made on the XmlJsonWriter, the team went with the latter. Now, not all types which implement IEDO will have this problem. Take the example below:
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyTypeV1 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
}
public class DCJS_and_IEDO_1
{
public static void Test()
{
MemoryStream ms = new MemoryStream();
MyTypeV1 v1 = new MyTypeV1 { str = "hello" };
DataContractJsonSerializer dcjs1;
dcjs1 = new DataContractJsonSerializer(typeof(MyTypeV1));
dcjs1.WriteObject(ms, v1);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
ms.Position = 0;
v1 = (MyTypeV1)dcjs1.ReadObject(ms);
Console.WriteLine(v1);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
}
}
The class MyTypeV1 can be safely serialized / deserialized – as long as the extension data is empty, nothing happens. Even when the extension data is not empty, as long as the type namespace is empty, this will also work fine:
[DataContract(Name = "MyType", Namespace = "")]
public class MyTypeV1 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
}
[DataContract(Name = "MyType", Namespace = "")]
public class MyTypeV2 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
[DataMember]
public int dc;
}
public class DCJS_and_IEDO_2
{
public static void Test()
{
DataContractSerializer dcs2 = new DataContractSerializer(typeof(MyTypeV2));
MyTypeV2 v2 = new MyTypeV2 { dc = 123, str = "hello" };
MemoryStream ms = new MemoryStream();
dcs2.WriteObject(ms, v2);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
ms.Position = 0;
DataContractSerializer dcs1 = new DataContractSerializer(typeof(MyTypeV1));
MyTypeV1 v1 = (MyTypeV1)dcs1.ReadObject(ms);
// v1 now has ExtensionData populated with 1 element; we can serialize it and it will be output
ms.SetLength(0);
DataContractJsonSerializer dcjs1 = new DataContractJsonSerializer(typeof(MyTypeV1));
dcjs1.WriteObject(ms, v1);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray())); // will output {“dc”:123,”str”:”hello”}
}
}
Now, once the type contains a non-empty namespace, that’s when we start seeing the issue:
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyTypeV1 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
}
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyTypeV2 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
[DataMember]
public int dc;
}
public class DCJS_and_IEDO_3
{
public static void Test()
{
DataContractSerializer dcs2 = new DataContractSerializer(typeof(MyTypeV2));
MyTypeV2 v2 = new MyTypeV2 { dc = 123, str = "hello" };
MemoryStream ms = new MemoryStream();
dcs2.WriteObject(ms, v2);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
ms.Position = 0;
DataContractSerializer dcs1 = new DataContractSerializer(typeof(MyTypeV1));
MyTypeV1 v1 = (MyTypeV1)dcs1.ReadObject(ms);
// v1 now has 1 member in the extension data: "dc", in the "https://my.namespace.com" namespace.
// It cannot be serialized with the DCJS anymore
Console.WriteLine(v1);
ms.SetLength(0);
DataContractJsonSerializer dcjs1 = new DataContractJsonSerializer(typeof(MyTypeV1));
// this will throw
dcjs1.WriteObject(ms, v1);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
}
}
Possible solutions (or workarounds)
So I hope you can understand the origin of the problem. Now what? One possible solution (as shown above) is to stop using namespaces (or using empty namespaces) for types which are to be consumed by the DCJS. This may work on some cases, but in most of them the types have a certain namespace for a reason, and removing them can be seen as a hack.
Another option (which is the one I’ve used when faced with this issue before) is to simply disable the IExtensibleDataObject from types when using them in JSON. The JSON serializer has one flag which allows us to do exactly that. The type still implements IEDO, and when used within the (XML) DataContractSerializer it will work just as before. However, when used with the JSON serializer, it will simply be ignored. The code below is the same as the previous one, with only the line where the DCJS is created changed (the parameter in red is the “ignoreExtensionDataObject”):
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyTypeV1 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
}
[DataContract(Name = "MyType", Namespace = "https://my.namespace.com")]
public class MyTypeV2 : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
[DataMember]
public string str;
[DataMember]
public int dc;
}
public class DCJS_and_IEDO_4
{
public static void Test()
{
DataContractSerializer dcs2 = new DataContractSerializer(typeof(MyTypeV2));
MyTypeV2 v2 = new MyTypeV2 { dc = 123, str = "hello" };
MemoryStream ms = new MemoryStream();
dcs2.WriteObject(ms, v2);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
ms.Position = 0;
DataContractSerializer dcs1 = new DataContractSerializer(typeof(MyTypeV1));
MyTypeV1 v1 = (MyTypeV1)dcs1.ReadObject(ms);
// v1 now has 1 member in the extension data: "dc", in the "https://my.namespace.com" namespace.
Console.WriteLine(v1);
ms.SetLength(0);
DataContractJsonSerializer dcjs1 = new DataContractJsonSerializer(
typeof(MyTypeV1), null, int.MaxValue, true, null, false);
// this will work – but the extension data member (dc) will be lost.
dcjs1.WriteObject(ms, v1);
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
}
}
I hope this will help people get unblocked when facing this issue. If that workaround doesn’t work, please let me know and I’ll investigate it more.