The web request API's in Windows Phone 8 SDK only support processing text from a web response in UTF8 or UTF16 encoding. This can be a problem if you need to work with older web servers which respond with text content encoded using other character sets.
Fortunately, the HttpWebRequest and HttpClient classes provide methods for accessing the response payload as raw binary stream which allows your application to decode the text, into a Unicode string, using custom code.
In this blog I will present two examples of custom methods for converting a raw binary array into a Unicode string. I will then show an example which uses HttpWebRequest to read the raw binary of a response and then uses the functions from first to examples.
Converting to string in managed code using a lookup table.
The first example I will show is using a simple lookup table which tells us the Unicode character corresponding to a given byte value from a code page. In this example I chose to use code page: windows-1251.
Using the data from the code page definition I created a table which contains 256 values, one for each possible byte value:
I then created a function: “StringFrom1251” which loops through the buffer building a Unicode string:
This function takes a byte array as input and returns a Unicode string. This is fairly straight-forward but requires a separate lookup table for every code page you want to support.
Converting to string using a native library and Win32 API
In many cases, Windows Phone 8 may already support the conversion from a give code page to the Unicode string natively but in order to leverage this in a Windows Phone 8 managed application you need to write a Windows Phone Runtime Component which you can call from your managed code, which is what I will show next.
The “Windows Phone Runtime Component” project template can be found in the “Windows Phone” projects group under the “Visual C++” language templates.
In the sample below I gave my project the name “NativeHelper” so the file names and namespace all refer to NativeHelper.
After creating the project I first added an include reference to the header file “agile.h” which gives me all the definitions my code needs to perform the actions I require.
Next I edited NativeHelper.h to define my class:”StringConverter” and one method: “GetUnicodeString”. GetUnicodeString needs to return a string, obviously, it also needs to know what code page to use for the conversion, and it needs the array of bytes which need to be converted. The result look like this:
Note: I declared both as “static” because the class does not need to maintain any instance data and making the function static allows me to use the method without creating an instance of the class.
After saving the changes to “NativeHelper.h”, I open NativeHelper.cpp to implement my GetUnicodeString method.
The GetUnicodeString method will use the Win32 API: MultiByteToWideChar… but you need to call this API twice, the first time you call with a null output buffer in order to get the buffer size required to hold the result.
You then allocate the buffer and call a second time to get the actual result. Finally, you need to take that result and use it to initialize a “Platform::String type which can be passed back to the managed code. The result looks like this:
Getting the byte array from an HttpWebRequest and Testing the code
To test the two methods I wrote a managed function which takes a string representing the Uri where the web resource can be found and a uint for expected code page. If the code page is 1251 I call the managed method otherwise I call the native method.
Upon completion my code calls a function called “UpdateStatus” which simply puts the resulting string into a TextBox control on my page, ex:
I have left some peripheral actions out of these examples in order to minimize clutter which might distract from the key objective. There are several things you, the reader, could do to expand upon this code such as checking the ContentType or ContentEncoding headers in the response for hints to the character set, instead of or in addition to, specifying the content type ahead of time. You could also use HttpClient instead of HttpWebRequest.
UPDATE: I corrected some logic errors in the native code sample related to the "output" string.
FYI: Don’t forget to follow the Windows Store Developer Solutions team on Twitter @wsdevsol. Comments are welcome, both below and on twitter.