“Encountered unexpected character 'ï'” error serializing JSON

When trying to deserialize JSON-encoded data from a file using the DataContractJsonSerializer, you may run into the following error:

System.Runtime.Serialization.SerializationException: There was an error deserializing the object of type
ConsoleApplication1.Person. Encountered unexpected character 'ï'. --->
System.Xml.XmlException: Encountered unexpected character 'ï'.
   at System.Xml.XmlExceptionHelper.ThrowXmlException(XmlDictionaryReader reader
, XmlException exception)
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadAttributes()
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadNonExistentElementName (StringHandleConstStringType elementName)
   at System.Runtime.Serialization.Json.XmlJsonReader.Read()
   at System.Xml.XmlBaseReader.IsStartElement()
   at System.Xml.XmlBaseReader.IsStartElement(XmlDictionaryString localName, XmlDictionaryString namespaceUri)
   at System.Runtime.Serialization.XmlReaderDelegator.IsStartElement(XmlDictionaryString localname, XmlDictionaryString ns)
   at System.Runtime.Serialization.XmlObjectSerializer.IsRootElement(XmlReaderDelegator reader, DataContract contract, XmlDictionaryString name, XmlDictionaryString ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalIsStartObject(XmlReaderDelegator reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContract Resolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   --- End of inner exception stack trace ---
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
    at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
   at ConsoleApplication1.Program.Main(String[] args) in e:\AzureSamples\JSONSerializer\ConsoleApplication1\ConsoleApplication1\Program.cs

One of the reasons for this could be that the input file that contains the JSON-encoded data is created with a binary encoding or it has a Byte Order Mark(BOM) byte sequence for a binary encoded file.
For e.g. The UTF-8 representation of the BOM is the byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.
Note: You will see this if you created a .JSON file(or a binary file) using visual studio.

To verify this, you can open the file using a binary editor like visual studio. To open the Binary Editor, first choose File | New | File from the main menu, select the file you want to edit, then click on the drop arrow next to the Open button, and choose Open With | Binary Editor.

image

image

image 

One way to solve the issue is to delete the BOM byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.

https://connect.microsoft.com/VisualStudio/feedback/details/356750/datacontractjsonserializer-fails-with-non-ansi-characters has the following which might be relevant:

Please note that DataContractJsonSerializer only supports the following encodings: UTF-8, UTF-16LE and UTF-16BE. For this to work, you need to do something like:     
     
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(panoramio));
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(PhotoData));
PhotoData pd = (PhotoData)dcjs.ReadObject(ms);