Designing "Time Zone conversions"


The BCL Team has been spending a lot of time investigating on how to provide richer support for Time Zones. As the PM who owns System.DateTime, I am tasked with figuring out the scenarios that are important for our developers whose customers needs to deal with the changing of time zones.


One of the most requested feature from our customers was to add the ability to convert from one time zone to another time zone. This is a relatively straight forward request, but things are never as easy as it seems. You would expect doing a conversion like this will require an API along the lines of:


DateTime TimeZone.Convert(DateTime dateTime, TimeZone src, TimeZone dest);


By just reading this, it seems quite straight forward. Given a certain DateTime, you convert it from the source time zone to the destination time zone.


E.g. My dog, Mojo, was born in Kenmore, WA (Pacific Standard Time – PST) on April 15th 2006 at 2am. If I want to find out Mojo’s birth time in New York, NY (Eastern Standard Time – EST) I will need to do something like this:


// My dog’s birthday
DateTime mojoBirthTime = new DateTime(2003, 4, 15, 2, 0, 0);
DateTime mojoNYBirthTime = TimeZone.Convert(mojoBirthTime, PST, EST);


You would expect, mojoNYBirthTime to be “April 15th 2003, 5:00am”. Simple!


However, haven’t we forgotten about the DateTime.Kind enum?? What if we did this:


DateTime mojoBirthTime = new DateTime(2003, 4, 15, 2, 0, 0);
DateTime mojoLocal = new DateTime(mojoBirthTime.Ticks, DateTimeKind.Local);
DateTime mojoEstBirthTime = TimeZone.Convert(mojoLocal, PST, EST);


If my machine’s local time zone is in Mountain Time (MST), then should I convert from PST, MST or what??? Similarly, what if:


DateTime mojoBirthday = new DateTime(2003, 4, 15, 2, 0, 0);
DateTime mojoLocal = new DateTime(mojoBirthday.Ticks, DateTimeKind.Utc);
DateTime mojoEstBirthday = TimeZone.Convert(mojoLocal, PST, EST);


Ah.. something as simple as a time zone to time zone convert doesn’t look so simple anymore. Should we respect DateTime.Kind, should we respect the passed in time zone or should we throw in this case? Things are never as simple as it seems. Does it happen to you too?


Additionally, with the US Time Zone changes coming into effect in 2007, (Per Engergy Policy Act of 2005: “Daylight Saving Time will start on the second sunday in March instead of the first Sunday in April, and will end on the first Sunday in Novemenber instead of the last Sunday of October”) the need for supporting changing time zones rules becomes more apparant. Though US is only experiencing these changes for the first time since 1986, many other countries (like Isreal) have to deal with daylight savings changes on a yearly basis. Vista has introduced the “Dynamic Time Zone Information” API to handle the changing of daylight saving time, in the managed code side of things, we are looking into ways of adding support. 


What do you think of Vista’s DYNAMIC_TIME_ZONE_INFORMATION API?


 


Comments (18)

  1. MichaelGiagnocavo says:

    Ignore the Kind enumeration if you specify it in convert function. It is quite non-intuitive to say "Convert this time, (which happens to be MST), from PST to EST."

  2. Des says:

    Day light saving is the single biggest cause of trouble for me as a developer. Let me explain, I work in Brisbane, Australia and our sales team is in Sydney, Australia. Though both locations are +10, Sydney has daylight savings +1 and Brisbane doesn’t.

    Because the server the web application runs on is located in Brisbane, we must store time, zone and location so that we can work out the correct time from the users point of view.

    The problem increased this year because of daylight savings changes caused by the Commonwealth games.  Which meant we had to go and update all the software that we have deployed.

    If you could solve this type of Time Zone/Daylight savings issues it would make my life a a lot easier.

  3. Jonathan Allen says:

    I agree. I suspect that most people even know that Kind exists. And I’m sure it isn’t used properly…

    time2 = time1.AddHours(3);

    Did I just convert the time from PST to EST?

    Or did I just want a timer to go off three hours from now?

    —-

    My suggestion is to just bite the bullet and create a real date-time class. Perhaps you can call it DateTimeZone.

    Unlike DateTime, DateTimeZone would fully account for time zones in all operations.

  4. Oren Novotny says:

    Why can’t the DateTime have a TimeZone as part of it?  A Date/Time can’t really exist without the context of a TimeZone.  By default, the current behavior would be to use the system’s time zone.  If a UTC time is parsed from a string, it’s generally converted to local time.

    It would elimiminate most of the confusion if a DateTime *always* had a timezone associated with it.  If a DateTime were already UTC, then a .ToUniversal() call would essentially do nothing.  Likewise, if it’s already the current time zone, then ToLocal would do nothing.  

    That way when you do conversions, you’re really only specifying what you want to convert to, not from since the from is implicit.  

    If I have a DateTime that’s EST in March and I do an AddMonths(5) on to it, then the resulting DateTime should be + 1 hour and EDT — so that the time is equivilent.  The definition of equiv should always be that the UTC value is the same.  

    Would that be possible?

  5. I’d agree with Michael – if you’re calling the static timezone.convert method, then surely the passed in date should be considered timezone independant, and it’s kind ignored.

    Perhaps a couple of overloads to allow different things.

    Timezone.Convert(DateTime date, TZ dest) should convert date to Dest, paying attention to the date’s kind, but when you provide source and dest, you presume that the kind is to be ignored.

    Also cool would be instance methods – add a convert method to the DateTime object that converts Me to the passing in time zone:

    Dim oldTime as DateTime = Now

    Dim newTime as DateTime = oldTime.Convert(PST)

    I don’t think I have any decent suggestions, but I can certinaly appreciate your problem now. Anything I can think of would include breaking back compatibility with previous version, which I guess you’d like to avoid if possible 🙂

  6. KathyKam says:

    Actually, we are investigating on how we can provide something like a DateTimeZone class. I’ll post my thoughts about it in a later blog.. but interesting design questions arises too! How should the “DateTimeZone” class be time zone aware? Time zone is geographical, political, and in some areas, even religious. Simply storing “PST”, “EST” is insufficient, should we store the whole time zone rules (including historic values)? If we do that, what if the time zone for that location changes by inventing a new one? i.e. If USA invented a new time zone for Utah, called “UST”? Should we store the time per location then, i.e “2am in UT, USA”? Again, there is always a state that is not quite so simple, e.g. Indiana is split between “CST” (areas close to Chicago wants to follow IL’s “CST” time zone and areas close to Ohio wants to follow Ohio’s “EST” time zone!) So in the extreme case, does this mean the DateTimeZone should really store longitude and latitude? 🙂  These design decisions are exactly what I want to highlight in this post, simple features are just not so simple afterall.

    That said, we are looking into designing it, and I’ll try and share my thoughts on it in coming weeks.

  7. Michael Giagnocavo says:

    Whoa… let’s get something straight — Time Zones are a way to offset UTC. The offset might change depending on all sorts of factors (some quite ridiculous, as you mentioned). But at the end of the day, they are simply that. Physical location has nothing to do with which timezone you choose, like it has nothing to do with which language you speak. Just because someone makes up a new time zone for Utah does not mean that everyone immediately and automatically has to adopt it. But even so, I don’t see how this should affect your library at all. You simply push a new DB update that has "UT" maybe with its subzones of "UDT" and "UST". Developers can take advantage of them then. Windows will also get the update, so people can change their clocks. On a per-application basis, they need to determine how to handle changing a time zone. It’s just like if they move from Australia to New Jersey, or when the government decides to change the DST rules. The library might provide some helper functions, but it should not try to automatically convert stuff for people.

    So, what people need to be able to do is go from an offset-time to UTC and vice versa. Of course, this needs to be a dynamic database and need to have historical information.

    Why is storing "Pacific Time" not sufficient? The timezone DB will have all the information on when Pacific Time is which offset (commonly referred to as PDT or PST). I don’t think there are any use cases for converting from PDT to EST, as long as Pacific and Eastern share the same daylight savings switch. And if they don’t then you’re really converting from "Pacific" to "Eastern" — the rules should just be handled.

    After you can go from UTC to local, then going from local to local is just going local to UTC to local. But you gotta stop people from saying something like "July 4, 2006, 5:00AM PST" — that’s not a valid combination (it’s PT or PDT). (Maybe July 4, 2010 will be PST if the government decides to change it, and then it’d be acceptable.) If someone really wants to store date times in a made up/invalid time zone, make them supply their own time zone data rules.

    The most difficult part that I see is deciding what to do when someone gives you a local time and only tells you "Pacific". If the time is an ambiguous time due to DST rollback, you need to get them to clarify. But that’s easy enough to resolve: either have them give you UTC, or allow them to tell you which "sub zone" that time was in. Since the system clock already knows if its on the DST shift, I think in most cases this can be handled transparently.  

  8. KathyKam says:

    Hi Michael,

    I love discussions! Thanks for the feedback! Let me answer your question, before I dive in..

    Why is storing “Pacific Time” not sufficient?

    Storing just “PST” is insufficient, because “PST” is simply an ID for you to look up the actual time zone rules. Different machine can have different DB. So your actual time will be different if you only store the ID.

    E.g.
    Machine A has the following PST rule:
    Offset = -8 hours
    Daylight saving offset = -7 hours
    Transition Start Date: 1st Sunday of April
    Transition End Date: Last Sunday of October

    Machine B has the following PST rule:
    Offset = -8 hours
    Daylight saving offset = -7 hours
    Transition Start Date: 2nd Sunday of March
    Transition End Date: 1st Sunday of Nov

    Then.. March 31st 6:00pm (PST) will represent different time for Machine A and Machine B. Conversions between UTC and other time zones will result in different values.

    “Physical location has nothing to do with which timezone you choose,”

    Physical location has everything to do with which time zone you choose. Our time zones are being differentiated by location! People doesn’t care about the time zone they are selecting. They only care about the time of day it is in a particular location. When I call home to Hong Kong, I don’t care what time zone they are in. All I care about is that I don’t wake my parents up at 3am in the morning. Users want a easy way to deal with time zone.Time zone gives people headache. Even the best of us! 🙂 The challenge is in how much we encapsulate and how much flexibility we allow our users to have. Do our developers need to know about the actual rule changes? How much flexibility do we give them? Since we’re the BCL, and the “B” really stands for “broad variety of users”. This is the design challenge I am trying to highlight in this post.

    Just because someone makes up a new time zone for Utah does not mean that everyone immediately and automatically has to adopt it.

    If the government changes the time zone on Utah, EVERYONE will adopt and use it. Yes, if some random person does this, no one would care, but if our government changes it. It will make a difference. We know our governments changes time zone, Australia is doing it, USA is doing it …etc.

    What do you think?

    Cheers,

    Kathy

  9. Oren Novotny says:

    One thing that would be very helpful would be if you could release a shared-source implementation of these new DateTimeZone classes so that we can start using them sooner rather than later.

    Many of us are dealing with these issues regularly and we’d rather not have to wait a long time until a BCL 3 is officially released.  

    Another thing — if you’re going to create a new DateTimeZone class rather than extend the existing DateTime (which would be preferable considering how many places DateTime is used), the new DateTimeZone class should have implicit conversion operators for DateTime.  That would enable a more seamless migration from one type to the other.  Since DateTime really stores its value as ticks from some constant UTC time, it should be possible to convert seamlessly.

    As far as some of your comments regarding where to store time zone rules, I think that it’s a fair trade-off to rely on the local system having an up-to-date conversion table.  That way you wouldn’t have to store the rules on the type thereby making it incredibly huge.

    In a majority of the systems that need time zone conversion it’s because we’re exchanging data with a remote system.  Almost by definition that means both systems are on a network and that means they’re able to get updates.  

    –Oren

  10. Oren Novotny says:

    Sorry, I was unclear in my last comment — extending the existing DateTime class is preferable to creating a new class since DateTime is already in wide use and it would be a huge effort to replace one type with the other everywhere.  

  11. Hi Kathy, thanks for writing back! I love discussions too.

    "

    >>Why is storing "Pacific Time" not sufficient?

    >

    >Storing just "PST" is insufficient, because "PST" is simply an ID for you to look up the actual time zone rules. Different

    >machine can have different DB. So your actual time will be different if you only store the ID.

    >

    >E.g.

    >Machine A has the following PST rule:

    >Offset = -8 hours

    > <snip>

    >Then.. March 31st 6:00pm (PST) will represent different time for Machine A and Machine B. Conversions between UTC and other

    >time zones will result in different values.

    "

    **

    But that’s what we have now. If someone doesn’t install a time zone update, then if I tell them "hey, meet you at 5PM Pacific

    Time", they’ll show up at a different UTC time than I would expect. I don’t think it’s unreasonable to have to have people to

    install updates to get correct time display. As you pointed out, the rules change year to year (sometimes as fast as a

    month), so the database will be large, and we can’t be attaching that every time.

    **

    "

    >"Physical location has nothing to do with which timezone you choose,"

    >

    >Physical location has everything to do with which time zone you choose. Our time zones are being differentiated by location!

    "

    **

    OK, I’ll admit I chose some very poor wording. I meant to say that physical location alone cannot determine time zone.

    Someone could be travelling, be working with other users and hence have their time zone set differently, etc. On all our

    server software, we set the clocks to UTC, and store all dates in UTC (so we never get burned by DST).

    I wouldn’t be surprised if, at some point in the future, some political/religious group declares one time zone, and another

    declares another, for the same latitude and longitude! Indeed, seeing how countries are prone to dispute territories, I’d be

    pretty certain this will arise (if it has not already). My point is that even if we did have an XYZ associated with a date

    time, it would not give us enough information.

    **

    "

    >People doesn’t care about the time zone they are selecting. They only care about the time of day it is in a particular

    >location. When I call home to Hong Kong, I don’t care what time zone they are in. All I care about is that I don’t wake my

    >parents up at 3am in the morning. Users want a easy way to deal with time zone.Time zone gives people headache. Even the

    >best of us! 🙂 The challenge is in how much we encapsulate and how much flexibility we allow our users to have. Do our

    >developers need to know about the actual rule changes? How much flexibility do we give them? Since we’re the BCL, and the

    >"B" really stands for "broad variety of users". This is the design challenge I am trying to highlight in this post.

    "

    **

    I know the responsibility of being on the BCL must be awesome, and I envy you guys :). I think the main difference in

    viewpoints is what you’ve highlighted here. If you view it from the point of "a person just wants to have their appointments

    work", then you can take a lot of freedom. But, each app has to deal with this stuff its own way. For a simple appointment in

    Outlook (i.e., just a reminder to myself), it’s obviously really easy. Store your information in the local time and the OS

    takes care of you.

    However, how does an app differentiate:

    – Tied to another time zone

    – Tied to a specific offset

    – Tied to the local system time (I want to wake up 9AM. If I move around the world, I want it to be 9AM.)

    You can replicate these things in Outlook. If you are scheduling with n people, some in DST-enabled time zones, some not, and

    someone gets an update — what did they actually agree on as the time? I ran into this a year ago. I lived in a non-DST zone

    (UTC-6), and had agreed to meet at 11AM "MDT" (UTC-6). Come fall, what time were we supposed to meet at?

    In the Utah example — how does my time change from "Mountain Time" to "Utah Time"? If I manually select it, am I saying

    "hey, I moved from Colorado to Utah"? If it auto updates, how does it know that chaning the time is correct?

    I love it when Microsoft solves a hard problem for us (that’s why MS is so big, I think?). That makes devs love MS. But, we

    should have to enroll in this functionality explicitly (even if it’s only a few lines of code, or heaven forbid, a wizard

    ;)), rather than letting it happen "automagically". A good example is the whole SOAP DateTime problem, where, IIRC, the

    serialization always embedded a UTC offset, creating a lot of problems (I was hit by it a long time ago), since the time zone

    might have been agreed on explicitly by the other side.

    I’d suggest that the APIs easily allow us to detect when a time zone was shifted, and then manipulate them. The application

    then needs to determine if/how it wants to update data, and what that entails in its context. For instance, if my

    appointments just shifted because Political Leader decided that if we offset by two hours for DST, maybe I (the user) will

    say, no, this appointment is a picnic, so I need it to stay relative to the sun. Or, perhaps, the application needs to send

    out sync messages or who knows what.

    A thing I love about the .NET and the BCL is that we can do things many ways. And a lot of value of the BCL is making

    classes/methods that wrap up otherwise available functionality. File.ReadAllLines is a great example of this. So, in the

    concept of dealign with changing time zones, perhaps we should be able to store a DateTime with a few extra bits of data so

    that the application can indicate if there is a time zone, and if so, what meaning does that have in light of changing zones

    (i.e., Mountain -> Utah), updates (new DST rules), and so on. Then, things can "magically" be converted for the app, and the

    dev has just delivered a lot more value without a whole lot of deep thought about TZs – they just set a few enums or clicked

    something in a wizard that did it for them. Meanwhile, developers who know what they are doing can work with things

    themselves.

    **

    "

    >>Just because someone makes up a new time zone for Utah does not mean that everyone immediately and automatically has to

    >>adopt it.

    >

    >If the government changes the time zone on Utah, EVERYONE will adopt and use it. Yes, if some random person does this, no

    >one would care, but if our government changes it. It will make a difference. We know our governments changes time zone,

    >Australia is doing it, USA is doing it …etc.

    "

    Sure, but then one county in Utah might decide against it for whatever reason, with half of the people there going along and

    half not. This year, Guatemala arbitrarily decided to implement DST (they had been fixed on UTC-6). So, within a month or

    two, they moved from UTC-6 to UTC-5. Many people living in certain villages simply did not respect this change for many

    reasons (some called it "devil time"). Then again, those people don’t typically use Windows or .NET, so perhaps it’s

    irrelevant to the BCL team. But something interesting is that, AFAIK, there is no OS patch for this one-year DST for

    Guatemala, or the other countries that soon sorta followed suit (they picked different days to revert) — at least, I haven’t

    seen anything on Windows update about it, and my time zone list looks the same. Not sure exactly what this means to BCL

    either. If it’s not in the timezone DB, then it doesn’t exist?

    Anyways, my bottom line is, please please please ensure that by default, devs must specify what their intent is. If there is

    some awesome thing that BCL comes up with that just magically deals with time zones, then make us explicitly opt-in to that.

    Also, I just saw the new comments — the "DateTimeZone" class — that would just be like Time Zone Info, right? It’s not a different class to store datetime in (that’d be confusing and difficult)?

    Thanks a lot!

    -Michael

  12. John Glassman says:

    In Jonathan Allen  reply above, he brings up the issue of an implicit adjustment for DST to a dateAdd calculation. While I would find optional argument to support this in an interface very useful, I would caution against having any such behavior a default. Comparisons to the time line against fixed units of time are fundamental to many business rules and SLA’s (service level agreements) and critical to correct accounting and billing practices for countless purposes. Such a behavior may certainly make it simpler to produce checks to see if an end time properly falls withing a set turn around time as time stamps could be directly compared with the result of the DateAdd, DateDiff, or whatever interval a range is based on. However, the use of these functions are fundemtal to existing calculation that may also then use addition inputs to adjust for DLS or Time Zone differences, especily if UTC time is not availble in the source data. If I add 5 hours to a datetime value, I should always expect it to show me a result time that is exactly 5 hour ahead, regardless of the calendar date and DST settings. But adding an ApplyDST boolean, and a zone id as an option would really be keen for a lot of applications and would simplify the production of such code. Any chance for getting these sort of time zone transforms built into SQLserver and SSIS?

  13. Kathy Kam says:

    So, in my previous &quot;Designing Time Zone Conversion&quot; post, a few readers asked whether we can update System.DateTime…

  14. Steve says:

    I have been struggling with this issue for some time.  I have a scheduling utility that takes a TZ specific DateTime, and I need to convert that to local system time to know when to run.

    Also, it is possible to overload, inherit, partial, or by any means derive from the DateTime structure?  Then I could add some of the features I need.