Thoughts on Open XML in ISO


As we move forward with the standardization of the Office Open XML formats, it’s interesting to look at the motivations that brought us to this point, but also to think about what is still to come. We’ve wanted to provide folks with easier ways to work with our formats for years now, mainly because it significantly increases the value of Office documents when they are fully documented. An open format can integrate with business processes; databases; and workflows in a much simpler and more powerful way (for more on why we made the move to open formats, read here and here). That’s why we’ve worked so hard over the past 3 or 4 releases to invest in other formats like RTF, HTML, and XML. These new Open XML formats which will be the default format for Office 2007 (as well as work in Office 2000, XP and 2003) are the result of all that work. If you’ve read my blog at all you know that it’s been a serious evolution and a lot of work, and I’m really excited about the potential. We already have hundreds of thousands of external developers building solutions on top of the XML formats from Office 2003 which weren’t even the default formats so you can imagine how huge this move to a new default XML formats is.


One thing I’ve heard from a number of folks though is that they are wondering what the next steps will be for the formats once they are standardized. Well, ultimately that is up to the organization that has taken over the ownership and maintenance of the formats. We’re currently standardizing the formats at Ecma international, which would mean that Ecma (which consists of representatives from a large number of companies in the industry) would own the formats as well as determine how the formats evolve. There has also been talk though of taking the formats to ISO once they have been approved by Ecma, which would mean that if ISO chooses to adopt the Open XML formats the stewardship of the formats would be theirs. We’ve had a number of governments indicate that they would like the formats to be given to ISO, and it’s likely that after the Ecma approval that will be the next step.


A number of people have asked if the approval of ODF by ISO has an impact on the standardization of Open XML. I don’t believe so given that ODF and Open XML have two very different goals in mind. Open XML was designed around compatibility with the existing base of Microsoft Office documents. There are literally billions of documents that exist today in those binary formats, and the goal of Open XML is to allow for a seamless migration from those old formats into the new XML formats. This is a huge undertaking, and it’s the reason that the spec is so large. I think that given the obvious need for an open XML format that achieves these goals, and the fact that ODF was not designed for that purpose, it’s clear that there isn’t a direct conflict between the two formats and there is no reason ISO wouldn’t want to approve and steward both formats. Rick Jelliffe, who has a wealth experience with ISO and standards has two posts that clearly call this out:




“ODF, for example, will change in no substantive way in its ISO adoption. National body comments will be added to requests or requirements for future versions. The Ecma Open XML people, so far, are being far more concilliatory in this regard: they know that a Microsoft technology doesn’t have the presumption of innocence that a Sun format does, in the minds of many.


If Microsoft/Ecma/et al manage to demonstrate to the ISO member voters that Open XML had even a first round of openness at Ecma, that it has some different use from ODF, if it supports SC34 specs like RELAX NG, and is scrupulous in its partitioning of Windows-specific hooks to another layer or namespace, I don’t see any national body rejecting Open XML, frankly. Microsoft and Ecma still have work to do in this regard, but it is just the standard kind of technical-level education/discussion/wordsmithing/re-alignment that any specification should have. ”



“They are generating lots of media attention, FUD and lobbeying; but it ODF and Open XML both represent a victory for universal, ubiquitous, standard generalized markup, which is what SC 34 is in large part about. I see Gartner has estimated a less than 70% chance of ISO ratifying two XML office formats. What rubbish. I’ll know more next week.


Ultimately, it is not WG1 or SC34 that makes the decision. It is the national votes of each of the voting members of ISO: the national standards organizations like Standards Australia, ANSI, and so on. While local committees may feel that Microsoft has been conspicuous in their absense, so have the other big companies in recent years: the standards participation focus shifted to W3C and OASIS. But these committees are not stacked with anti-Microsoft (or anti-Sun) people, but with organizations who need good interchange and also need an XML retrieval for legacy documents in proprietary formats (.DOC, etc.). So I find it very difficult to agree with Gartner’s 70%; I’d put it the other way, with a 70% likelihood of success, at least.


ISO is not an anti-monopoly court. It is there to help people who want to agree on technology, providing procedures, forums and a publishing house.


But the issue of having two office standards is a fair one. I think all Microsoft needs to do is to distinguish Open XML from ODF adequately and prove that it has a credible alternative constituency who would not be served well by ODF. That there is overlap is immaterial if there is a significant difference.”


For anyone who has played around with the XML formats, I’m sure you’ve seen that we really took seriously our goal of minimal user impact in the move to default XML formats. This included things like performance (which I’ve already briefly touched when I talked about spreadsheetML, tag lengths, and shared formulas), as well as full compatibility with the existing base of Office documents. I’ve just recently started to show some basic examples of where ODF just doesn’t come through in terms of compatibility, such as with formulas, numbering formats, and highlighting (and these were just the first three things I came across… I have a growing list that I’ll talk about over the coming months).


-Brian

Comments (18)

  1. Brian,

    let’s say after Ecma, you submit the format to ISO. I don’t know how the ISO process works, but I am wondering whether the format could change during the ISO process? Meaning, could the ISO process itself alter the formats or would ISO simply accept the format as is from Ecma?

    Thanks,

    Patrick

  2. BrianJones says:

    I’m not sure Patrick. Looking at Rick’s comments and following the ODF process, I think that any changes are usually around the documentation itself and clarifying various things. That said, Rick has a lot more experience with how ISO works, so you should check out his blog and post questions there around ISO. I’m still wrapped up in the Ecma processes and shipping Office 2007 at this point :-)

    -Brian

  3. mcm says:

    Since ODF has been ratified as an ISO standard would there be any chance that Microsoft Office will support the format too in addition to its own new default XML format?

  4. BrianJones says:

    – mcm,

    We won’t provide ODF support natively in Office 2007. I’m sure we’ll see a number of 3rd party add-ins for Office that allow it to read and write ODF though (there was already one reported by one of those OpenDocument committee sites).

    As far as whether or not it will be integrated directly into the product in a future version, I can’t say as we haven’t yet planned what we’ll do with the next version. That will really depend on whether or not there is a significant enough demand from our customers. We’ve had plenty of customers ask for XML formats, but most of them don’t really care whether it’s ODF or not. The main thing they care about is that the format supports all of their existing documents, so there is no migration pain (which ODF does not do).

    -Brian

  5. Sam Sethi says:

    Google Spreadsheets was announced this morning with a terse comment about supporting Microsoft Excel file formats.  Probably CSV and XLS to begin with but no support for XLM. According to Google I could search for XLS files and simultaneously edit and chat about them using Google’s instant messaging program.

    I’m sure Google spreadsheets will support ODF and  eventually Open XML when it becomes an ISO standard.  

    Then using Google’s Gdata (Google’s Atom based xml query extension) it will make it even easier to store, find manipulate XML based spreadsheets whether they came from OpenOffice or Microsoft Office.

    So to remain competitive I guess MSN/Live (desktop) Search will have to support indexing of OpenOffice ODF files. Maybe Microsoft will create Excel lite for free as a Live Gadget or Groove add-in?

  6. Adam says:

    "They are generating lots of media attention, FUD and lobbeying; but it ODF and Open XML both represent a victory for universal, ubiquitous, standard generalized markup, which is what SC 34 is in large part about."

    I don’t see how this can be interpreted other than as FUD itself.

    OpenXML does not represent a victory for a universal or ubiquitous format, as it has not been designed with re-implementation in mind.

    This is, in fact, the entire crux of the difference between the goals of the groups of people behind ODF and OOX. And until any MS can convince at least one other vendor to support OOX _completely_ on a system _other than Win32_, such that OOX documents can be created, edited, exchanged, modified and re-exchanged between MS Office and similar suites on those systems with zero major interoperability problems[0], claims of universitality or ubuiquitousness are basically unsupportable.

    Yes, OOX support for legacy DOC files is important. And the sorts of things that people will be able to do with their own documents in OOX that they can’t do while they’re in DOC format is a real bonus, and those advantages shouldn’t be downplayed.

    But I don’t see OOX being supported on hetergenous systems in anywhere near the same sort of timescale that ODF will be, and that has very little to do with the fact that it’s "made by Microsoft". In fact, due to the volume of OOX documents that are going to exist soon, I imagine that OOX support will be incredibly high on the priorities of the developers of other office systems. I just don’t think they’ll finish it anytime soon though.

    [0] minor implementation bugs on either side are, of course, expected to begin with.

  7. Thomas Lee says:

    Brian says "We won’t provide ODF support natively in Office 2007. I’m sure we’ll see a number of 3rd party add-ins for Office that allow it to read and write ODF though (there was already one reported by one of those OpenDocument committee sites)."

    This is dissapointing. To me, this is a backing off from  MS earlier committments to support open standards. Not only do customers not get PDF, but also no ODF.  All we are left with are propritary and private doc formats, not the ones customers I hear asking for.

    It feels like MS is sticking two fingers up at Europe and the Open Source community, while at the same time talking about delivering on customer requests and the value of open standards. The story simply sounds less than good here in Europe.

    I continue to believe MS should bite the bullet and do ODF properly. We all know that Office is going to have to save in downlevel formats for a long time to come (eg. RTF, .DOC, .XLS etc). Having another one is hardly going to be a big deal, now is it. Especially as the testing time for PDF is gone, you would have extra testing cycles.

    By not doing this:

    a) you just annoy the politicians who now have something further to beat you up with and/or

    b) others do it and do it badly causing more MS support calls anyway.

    Just like MS did not do an Netware client in NT 3.1 at first, but eventually had to, MS should just go do OpenDoc and let the market decide which "open" standard they want to adopt.

    My .02€ worth

  8. Sean DALY says:

    > We’ve wanted to provide folks with

    > easier ways to work with our formats for

    > years now, mainly because it significantly

    > increases the value of Office documents

    > when they are fully documented.

    If this is true, why are the proprietary binary formats still kept secret?

    Were Microsoft to open the file formats, non-Microsoft applications would have far less difficulty helping users get access to their own archival data.

    Unfortunately, Microsoft’s shareholders are far more important than Microsoft’s customers, which is why the world needs an open, unencumbered standard document format.

    Sean DALY.

  9. marc says:

    brian jones said: "We won’t provide ODF support natively in Office 2007.

    …As far as whether or not it will be integrated directly into the product

    in a future version, I can’t say as we haven’t yet planned what we’ll do

    with the next version"

    hurry up, do the planning !!  😉 :

    http://technocrat.net/d/2006/5/26/3790

    http://www.computerworld.com.au/index.php?id=954149621&eid=-6787

    http://www.egovmonitor.com/node/5402

    http://www.sept-solutions.de/English/newsdetail.php?id=37

    https://www.ebidsourcing.com/processPublicSolSummView.do?action=soltypeCd&docStatus=CLOSED&docUserId=3070&docViewType=CLOSED&docId=106466&doValidateToken=false&soltypeCd=ITP

    ( the last link is a ‘request for info’ about an ‘OpenDocument Format

    Plug-in for Microsoft Office Suite’ issued by the Commonwealth of Massachusetts )

  10. Gabe says:

    You better hurry up and add ODF support now, before some ISV beats you to it. Because if you wait 10 years and some ISV builds up a business around exporting as ODF, they’ll cry about anti-competitive behavior and try to sue when you go to implement the feature yourselves.

  11. Hmm … one thing very much in OpenDocument’s favor is its open process. Anyone can join the OASIS ODF TC and make contributions. Likewise, absent that, there’s a public comment form (and TC members read those comments).

    By contrast, I’m finding getting information on the new citation and bibliographiy supoprt in OXML (not well specified in the current ECMA draft), or submitting comments about them, quite difficuilt. Can you ping me Brian about clearing this up?

  12. Wesley Parish says:

    Just a few comments.

    Duplication of effort: there are now two different XML-based document file formats.  How much effort would have been needed for Microsoft to make sure that the relevant areas where you now find incompatibilties between ODF and Microsoft Office’s binary file formats, were in fact made compatible?  Or in other words, how much time could have been saved if Microsoft had chosen a different path?

    Open XML’s MS Windows dependencies: these are raised by ODF developers as a major problem with implementing Open XML in any non-Microsoft environment.  Assuming that Microsoft truly does wish to see Open XML implemented on competing products such as the crossplatform wordprocessor AbiWord, is Microsoft willing to take the steps necessary to assure open source re-implementors that it will not use those Windows dependencies against them?  I mean, it’s very nice that Microsoft has a covenant not to sue independent re-implementors of Open XML, but so far I haven’t read of any such covenant not to sue re-implementors of ActiveX, for example.  And that would be necessary to get an independent open source re-implementation of Open XML running on Linux or Solaris.

    I have suggested to Jason Matusow that Microsoft submit the MS Win32 API to ECMA and thence to OSI as a standard; he’s a bit dubious.  I suspect that this little detail here might be a perfectly adequate reason to do so.  Because Microsoft can’t make that sort of mistake too often – opening the door only to slam it on people’s feet – and expect to be trusted.

    And lastly, from your words

    "Open XML was designed around compatibility with the existing base of Microsoft Office documents. There are literally billions of documents that exist today in those binary formats, and the goal of Open XML is to allow for a seamless migration from those old formats into the new XML formats.",

    do I understand that Open XML is an authoritative documentation of the MS Office file formats dating back to MS Office 97?  If so, I expect it would be possible to develop ODF to close the compatibility gap by working from the Open XML documentation.

    Comments?

  13. steve says:

    Brian,

    I find the tone of your blog condescending.

    You make it sound as though ODF (the ISO standard) is some poor cousin which we should feel inadequate using next to your formats (which aren’t even available yet!).

    I use ODT files day in day out knowing that I or anyone who needs to will be able to open and edit them in the future.  I also have a wide range of choices of software to edit ODF files with. The least you could do is add support for ODF in Office 2007. I assume 2007 can open text files right?, rtf? What makes ODF less useful than either of those?

    "That will really depend on whether or not there is a significant enough demand from our customers"

    How many requests for ODF support would make "significant enough demand"?  

  14. marc says:

    Monday, June 05 2006, Brian Jones said ( http://blogs.msdn.com/brian_jones/archive/2006/06/05/618089.aspx#618437 ):

    "We won’t provide ODF support natively in Office 2007. I’m sure we’ll see a number of 3rd party add-ins for Office that allow it to read and write ODF though"

    Wednesday July 05 2006, Microsoft said ( http://www.microsoft.com/presspass/press/2006/jul06/07-06OpenSourceProjectPR.mspx ):

    "[Microsoft] announced the creation of the Open XML Translator project….A prototype version of the first translator added to Word 2007 will be posted today on the open source software development Web site SourceForge

    (http://sourceforge.net/projects/odf-converter)"

    Brian, please be sincere when blogging here, respect your readers

    why didnt you say: "i’m not allowed to say if we will or not provide ODF support"

    or

    "yes, we will provide ODF support in Office 12, we’ll see several MSOOXML-ODF translations tools sponsored from Microsoft"

    This is a marketing blog or a technical one? we must believe your words or read between lines?

    -Marc

  15. BrianJones says:

    Hey Marc, we’ve always encouraged folks to build add-ins to provide ODF support, and my statement above still holds true. Microsoft is not building ODF support into Office.  Instead, we made the decision recently to support an open source project to build an add-in (just like we’ve been encouraging folks to do for the past year).

    Sorry if it looks like I’m not being straightforward, but that was an honest statement. We are not going to build ODF support into Office. We were thinking about directly supporting an opensource project (which we’ve now decided to do), but I couldn’t mention it yet so I didn’t say anything.

    -Brian