“Encountered unexpected character ‘ï’” error serializing JSON

When trying to deserialize JSON-encoded data from a file using the DataContractJsonSerializer, you may run into the following error:

System.Runtime.Serialization.SerializationException: There was an error deserializing the object of type
ConsoleApplication1.Person. Encountered unexpected character 'ï'. --->
System.Xml.XmlException: Encountered unexpected character 'ï'.
   at System.Xml.XmlExceptionHelper.ThrowXmlException(XmlDictionaryReader reader
, XmlException exception)
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadAttributes()
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadNonExistentElementName (StringHandleConstStringType elementName)
   at System.Runtime.Serialization.Json.XmlJsonReader.Read()
   at System.Xml.XmlBaseReader.IsStartElement()
   at System.Xml.XmlBaseReader.IsStartElement(XmlDictionaryString localName, XmlDictionaryString namespaceUri)
   at System.Runtime.Serialization.XmlReaderDelegator.IsStartElement(XmlDictionaryString localname, XmlDictionaryString ns)
   at System.Runtime.Serialization.XmlObjectSerializer.IsRootElement(XmlReaderDelegator reader, DataContract contract, XmlDictionaryString name, XmlDictionaryString ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalIsStartObject(XmlReaderDelegator reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContract Resolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   --- End of inner exception stack trace ---
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
    at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
   at ConsoleApplication1.Program.Main(String[] args) in e:\AzureSamples\JSONSerializer\ConsoleApplication1\ConsoleApplication1\Program.cs

One of the reasons for this could be that the input file that contains the JSON-encoded data is created with a binary encoding or it has a Byte Order Mark(BOM) byte sequence for a binary encoded file.
For e.g. The UTF-8 representation of the BOM is the byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.
Note: You will see this if you created a .JSON file(or a binary file) using visual studio.

To verify this, you can open the file using a binary editor like visual studio. To open the Binary Editor, first choose File | New | File from the main menu, select the file you want to edit, then click on the drop arrow next to the Open button, and choose Open With | Binary Editor.




One way to solve the issue is to delete the BOM byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.

http://connect.microsoft.com/VisualStudio/feedback/details/356750/datacontractjsonserializer-fails-with-non-ansi-characters has the following which might be relevant:

Please note that DataContractJsonSerializer only supports the following encodings: UTF-8, UTF-16LE and UTF-16BE. For this to work, you need to do something like:     
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(panoramio));
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(PhotoData));
PhotoData pd = (PhotoData)dcjs.ReadObject(ms);

Comments (8)

  1. Carlos Ponce says:

    Thank you very much! I've been looking for days how to solve this issue and didn't found the answer, until now. Does this happen with ALL of the JSON files created through Visual Studio? It's a bit frustrating that the mistake was with the way the serializer encoded the data, not a mistake by myself (I'm a beginner in programming so that's the common thing). Is there a way to avoid this?

  2. D says:

    Thanks!  This helped me in a little test harness I'm using.  Very good tip.

  3. JoGe says:

    Thank you! Almost getting mad with encoding during a bigger project. And didn`t find anything about this kinky special problem. Very helpfull!

  4. Bahata says:

    I had the same problem. I solved it like this. I used free NotePad++ . I used the Encoding tool there and encoded the file using "Encode with UTF-8 without BOM". This one click approach resolved the issue.

  5. Christofer Toledo says:

    Muchisimas gracias!!! fue de mucha ayuda!! 😀

  6. Don Carlos says:

    Thank you! This was driving me crazy, in my case I was calling File.WriteAllText(unc, dataAsString, Encoding.UTF8) which was causing it to write in the UTF-8 BOM encoding.

    1. Thanks for you comments Don. Glad that it helped you.

  7. Good post. The Connect ticket workaround is definitely more robust.
    Since you’re starting out from a file, the end to end solution would look like this:

    using (var ms= new MemoryStream(Encoding.UTF8.GetBytes(File.ReadAllText(file))))
    foo= (Foo)(new DataContractJsonSerializer(typeof(Foo)).ReadObject(ms));

    BTW the original Connect link is dead but you can find it here: https://web.archive.org/web/20140610215024/http://connect.microsoft.com/VisualStudio/feedback/details/356750/datacontractjsonserializer-fails-with-non-ansi-characters

Skip to main content