How to Work Around Problems Serializing DateTime in XML [Anthony Moore]

  • Article

I wrote a while ago about problems with the way DateTime worked with features like DataSet, XML Serialization (WebServices) and XmlConvert. Basically, DateTime is always treated as a local time by these XML-based systems, which means that you get incorrect results if you use UTC or whole dates with them. What makes this worse is that it is actually a best practice to the UTC times rather than local times. People often don’t discover this problem until they are very invested in a technology and test their application with different tiers in different time zones for the first time.

 

At the time of the previous post, there was discussion about whether this would be fixed in a service pack. Right now, this is very unlikely to happen, because these problems are all corrected in Whidbey and we are getting close to it being released, so the value of a fix in servicing is much reduced.

 

The Beta 2 release will be available fairly soon. It will be a release of at least the quality level of V1.0 Beta 2. Many companies chose to deploy live applications on the V1.0 Beta 2 and many are choosing to do so with the Whidbey Beta 2, as the quality level will be very close to that of the final release. So, a work-around to consider for all of these problems is to adopt Beta 2 a little sooner than you were planning.

XmlConvert

There is a fairly easy work-around to this problem if you call XmlConvert directly. Instead of this:

 

String output = XmlConvert.ToString(dateTimeInput);

 

Call this:

 

String output = dateTimeInput.ToString(“yyyy-MM-ddTHH:mm:ss.fffffff”);

 

This is a valid XML date format, and is almost identical to what XmlConvert currently does, without the problematic local time zone. This is suitable for whole Dates and UTC dates, and you can use XmlConvert.ToDateTime on the other end.

 

It is slightly more correct to use this format for UTC times:

 

String output = dateTimeInput.ToString(“yyyy-MM-ddTHH:mm:ss.fffffffZ”);

 

However, this one won’t work with XmlConvert.ToDateTime on the other end as expected, because it will default to converting this to local time for you. If you want to use this format, use this call instead of XmlConvert.ToDateTime:

 

DateTime output = DateTime.ParseExact(stringInput,

                  “yyyy-MM-ddTHH:mm:ss.fffffffZ”,

                                      DateTimeStyles.AdjustToUniversal);

 

This is not so close to what XmlConvert.ToDateTime does. It calls ParseExact with a set of 24 different formats of which this is one. If you want to support multiple formats, pass in an array with the XML formats you want. Another option is to call XmlConvert.ToDateTime and then call ToUniversalTime on the result, but this has a reliability caveat (see below).

 

Web Services (XML Serialization)

 

It is harder to work around this problem in Web Services. Here are some options for using a UTC or a whole date with Web Services.

 

  1. Internally use the data as a DateTime, but serialize it either as a String using the format mentioned above or as an Int64 tick count.
  2. Call DateTime.ToLocalTime on the DateTime before putting it in the serializable class and call DateTime.ToUniversalTime after taking it out. This will effectively “cancel out” the adjustment, and can be used whether you are dealing with a whole date or a UTC time.
  3. For whole dates where you do not need to serialize the time portion, you can change the XML schema to indicate that it is a date rather than a date and time, which will change the format to “yyyy-MM-dd”, which will serialize correctly.
  4. For UTC times, you can have an extra property on the class that exists for serialization only. This work-around is outlined in this article.
  5. Make all machines use the same time zone.
  6. If you have a chance to pre-process the XML before it is sent out, you can manually strip out the time zone offset out of the XML text. For example, a typical XML date and time looks like this: “2005-01-28T03:14:42.0000000-07:00”. You can use a Regex to remove the “-07:00”. You do not need to re-inject anything on the other end, as no adjustment is made if there is no time zone information. Do not try to replace the time zone offset with “Z” or “+00:00”. While technically a more correct representation, the existence of time zone information will cause the serializer to do an extra conversion to local.

 

Obviously some of these work arounds are quite horrible. For Web Services I would recommend (1) for UTC times and (3) for whole dates. Options (2) and (4) have a reliability caveat explained below.

 

DataSet

 

DataSet is the hardest technology of the three to work around this problem. Some options:

 

  1. Change the column types to be Int64 or String.
  2. Call DateTime.ToLocalTime on the DateTime before putting it in the DataSet and call DateTime.ToUniversalTime after taking it out. This will effectively “cancel out” the adjustment, and can be used whether you are dealing with a whole date or a UTC time.
  3. Make all machines use the same time zone.
  4. Use Remoting to serialize the DataSet in binary form. This also has performance benefits. This KB article has an example.
  5. If you have a chance to pre-process the XML before it is sent out, you can manually strip out the time zone offset out of the XML text. For example, a typical XML date and time looks like this: “2005-01-28T03:14:42.0000000-07:00”. You can use a Regex to remove the “-07:00”. You do not need to re-inject anything on the other end, as no adjustment is made if there is no time zone information. Do not try to replace the time zone offset with “Z” or “+00:00”. While technically a more correct representation, the existence of time zone information will cause the serializer to do an extra conversion to local.

 

This is the most difficult situation, because all of these work-arounds have problems. Option (1) involves bypassing the database type system. Option (2) has a reliability caveat explained below. I would actually recommend (4) or (5) for this technology.

 

Reliability Caveat for Local Conversions

Some of these work-arounds involve conversion between the local time and UTC and others involve using UTC and ensuring that it is never converted. Some of the work-arounds involving local conversion are a little easier, but they should not be used if you have an application that needs to be 100% reliable 24/7. There is one hour per year where local clock times are inherently lossy, because there is an hour that gets repeated when you wind the clock back as you leave daylight savings time. In V1.0 and V1.1, the DateTime did not differentiate these two times, so even temporarily converting to local can cause data loss.

 

In the Whidbey release, the data format has been modified to ensure that local times are not lossy in this situation and can differentiate between these two times. Despite this, UTC is still the recommended way to persist times and do arithmetic on them, because it works across changes in the time zone and the arithmetic is not affected by daylight savings time transitions.