Politics behind standardization


Stephen McGibbon has an interesting blog post based on his observations of the politics behind standardization. He’s been involved in a lot of the ODF and Open XML discussions and started looking deeper into the reasons behind ODF going through ISO even though it was not yet feature complete.


I think there was a good amount of pressure to wrap things up and say that version 1.0 was ready to go. It will be interesting to see how much further they are able to get with version 1.2. It sounds like there will be a draft of that ready for review next summer and approved by OASIS in fall 2007 (http://lists.oasis-open.org/archives/office/200605/msg00005.html). I’m sure they will be able to make a lot of progress over the next year or so.


As I talked about earlier this week, it doesn’t make sense for us (Microsoft) to join the ODF committee in OASIS. In the original post some folks felt that I didn’t really clearly provide reasons why this was the case, but lower down in the comments I took another stab at trying to explain the reasons and I think most folks felt it helped to clear things up. Here is what I said (for those of you that don’t feel like reading through comments).


“Bruce, Wouter, Simon,
I’m sorry if it appeared like I didn’t answer the question. I thought I had actually made it clear why we weren’t participating directly in the OASIS committee, but let me try to clear it up.

We ultimately need to prioritize our standardization efforts, and as the Ecma Office Open XML spec is clearly further along in meeting the goal of full interoperability with the existing set of billions of Office documents, that is where our focus is. The Ecma spec is only a few months away from completion, while the OASIS committee has stated they believe they have at least another year before they are even able to define spreadsheet formulas. If the OASIS Open Document committee is having trouble meeting the goal of compatibility with the existing set of Office documents, then they should be able to leverage the work done by Ecma as the draft released back in the spring is already very detailed and the final draft should be published later this year.

To be clear, we have taken a ‘hands off’ approach to the OASIS technical committees because:  a) we have our hands full finishing a great product (Office 2007) and contributing to Ecma TC45, and b) we do not want in any way to be perceived as slowing down or working against ODF.  We have made this clear during the ISO consideration process as well.  The ODF and Open XML projects have legitimate differences of architecture, customer requirements and purpose.  This Translator project and others will prove that the formats can coexist with a certain tolerance, despite the differences and gaps.

No matter how well-intentioned our involvement might be with ODF, it would be perceived to be self-serving or detrimental to ODF and might come from a different perception of requirements.   We have nothing against the different ODF committees’ work, but just recognize that our presence and input would tend to be misinterpreted and an inefficient use of valuable resources.  The Translator project we feel is a good productive ‘middle ground’ for practical interoperability concerns to be worked out in a transparent way for everyone, rather than attempting to swing one technical approach and set of customer requirements over to the other.

-Brian”


Now, all that said, I think there are still plenty of ways we can help out the OASIS folks with the ODF format. The entire translator project is open source, so the conversion will be completely transparent and everyone will have the ability to benefit from what we discover as the transformations are built. In addition to that, as I’ve looked through our Ecma documentation, I’ve also been looking at the ODF spec as a point of comparison. As I come across areas that are either missing, or just not fully specified, I’ll be sure to point them out on my blog. That should help them in creating a list of areas to improve.


I think we will really see some good discussions over the summer and fall. The Ecma spec is really getting close to completion, and we of course still have a large number of ways for the public to comment on the spec. Now with the open source translator project we’ll all be able to clearly follow along with how the two formats compare and how you can go back & forth between the two.


BTW, for those interested in providing feedback on the Ecma spec, the main way to comment is via this e-mail address (mailto:ecmatc45feedback@ecma-international.org). You can also provide comments here on my blog and I’ll pass them on. Of course the best approach though would be to join us in Ecma!


-Brian

Comments (13)

  1. orcmid says:

    I find your approach very understandable and look forward to the long term benefits these contributions will make toward genuine interchange and preservation of digital documents.

    What I find intriguing about "framing the debate" (something people who talk about FUD and framing the debate  are doing when they complain about others doing it), is that most of the visible ODF activity is around advocacy and treats the heavy lifting of creating a technical interoperability and interchange regime as a done deal.  This reminds me of the rumored (but plausible, in my eperience) historical IBM approach to persuading management and letting the techies and operational folk eat the consequences.  OK, end of the snarky part.

    I commend you for keeping your attention on the serious work that it takes to create an interchange ecology, and the translator is a great offering in that regard.

    I notice that a lot happened at the ECMA TC45 and some of it sounds pretty material in our looking at how formats can be repurposed and inter-converted.  Is there a new draft for download and I’ve just missed it?

  2. JayV says:

    Hey Brian,

     The e-mail ‘ecmatc45feedback@ecma-international.org’ appears to be private, and it requires permission to send mail to that address.  Here’s the email that I tried to send (regarding the <sectPr> performance that I commented on before).  If you can pass this on to ECMA, I’d appreciate it.  Thanks.

    Subject: WordML <sectPr> diminishes performance

    The section properties in WordML (<sectPr>) are defined to occur at the end of a section, but in order to improve document consumer performance, wouldn’t it be advantageous to either have the section properties at the start of a section or even group all section properties in a separate XML part?

    For example, when section properties occur at the end of a section, a consumer is required to read the entire section before correctly displaying any content because things like margins, page size, headers, and footers are unknown.  For large documents that only contain a single section, this means the whole document must be read before displaying anything.  On the other hand, if the section properties are available at the start of each section, the consumer can immediately start displaying content, and it eliminates the need to read through each section (and ultimately, the whole document) twice.

    I don’t see any real benefit to having the section properties at the end of a section.  If you could respond by e-mail(user:jvanriper at, domain:stellent dot com) with either an explanation of why this was done or whether you are considering taking action to correct this, I would greatly appreciate it.

    Thank you,

    Jay

  3. A says:

    Brian, please post the reply here too 🙂  I’ve also been wondering about this.  As Jay said, we have to parse the entire document, gather up all the section properties into a list, and then start all over again.  It isn’t difficult to implement, but it is a performance hit.

  4. Brad Corbin says:

    Why put the formatting at the end of the section instead of the beginning? Ah, that’s an easy one.

    Because that’s how MS Word does it.

    This isn’t the first time we’ve seen the standard bend toward the methodology used by the MS application team instead of adopting what some might view as an improvement in the format.

    Witness the discussion a while back about page borders in Word: Why have a spec that lists out 295 different legacy page border styles by name, instead of a 1-page specification that says:

    "Use the following syntax for displaying a graphical page border. The repeated elements of the page border should be included as resource files, designated with a type of…"

    Why? Because doing so would create more work for the MS Word programmers, who are used to just saving "BorderStyle = Baloons" in their word Docs.

    And Office 2007 has a deadline, too–you can’t really release the product if the specification for OpenXML is still changing, can you?

    Reminds me of someone who said:

    "I think there was a good amount of pressure to wrap things up and say that version 1.0 was ready to go."

    Who was that again? 🙂

  5. BrianJones says:

    Jay,

    I’m not sure why the e-mail address didn’t work for you. I’m going to check with the Ecma folks and see if they can fix the problem.

    I’ll also bring up that issue around section properties with the Ecma TC and see if there is anything we can do about it. I remember discussing this earlier and there was a reason why we chose to put them at the end rather than at the beginning. You raise some good points though, so I’ll look into this a bit more and see if it’s a better trade-off to put them at the beginning.

    Brad,

    I think there is a difference between making design tradeoffs based on targeted delivery dates, and leaving a significant portion of the spec unspecified. You can always take more time trying to reach an optimal design, and at some point you have to draw the line.

    The ODF spec is a much different story though. They left large areas of the spec completely unspecified in order to rush it through ISO. That (in my opinion) was the wrong tradeoff.

    -Brian

  6. Brian – While I appreciate the difficulty in both solidifying a standard and getting a product out, I do think there are valid points made about parts of the Open XML spec that seem designed simply to make life easier for the existing MS Office products.  Of course, there is no reason to make life difficult for them on purpose, but with a bit more work, the spec would be more flexible and resilient, so that when the MS Office developers want to put in new features later on, they won’t be constrained either.  In other words, even Microsoft is likely to be hurt in the long run if the specs aren’t made a bit more generic.  Might it be possible to fence off certain areas intentionally as "optional" and provide an alternative that is more generic.  I am just afraid that Microsoft is serving its short term needs without enough thought given to its long term needs.  ODF has some of the same issues, but from the opposite direction, since it does not include some features that would make it easier for MS Word to maintain its current features in a document expressed in ODF.  The obvious reality is that there are lots of Word documents (not Open XML documents, but .doc documents), and making ODF cognizant of that would actually strengthen the standard, in my opinion.

    – Ben

  7. Wesley Parish says:

    It also seems there are a number of versions of MS Works out there and their file format is not recognized by MS Office 9x/2kx.  I know this because I tried to open an MS Works Word file in MS Office 2K Word for a customer in the CTLC cybercaf I do voluntary work at, and it didn’t work.  The truly terrible thing was that it was the poor woman’s CV/resume and she needed it rather desperately to apply for some jobs.  I felt guilty for not being able to open the file and print it for her.

    Is MS Open XML going to support MS Works file formats?  This is one thing that nobody’s bothered to ask yet.  It’s as if MS Works users are second-class citizens.

  8. BrianJones says:

    Ben,

    This actually is a pretty important part of the overall design for the file format. It’s extremely flexible if anyone wants to come along and extend it. It’s also flexible in the way that most of the spec is optional, and it’s up to a developer to decide which features they want to support.

    In terms of the extensibility, there are a couple different layers. You can add your own tags and attributes in your own namespace, and just specify that anything in that namespace is either ignorable, or should be understood. You also have the ability to provide a choice block, and each choice can have different representations. So if you want to add some new stuff in your own namespace, you can provide that as one choice, and then have a more basic representation as a fallback. We view the extensibility stuff as being extremely important, because we want to make sure that we can add new features to the next version of the spec, and not have that break Office 2007’s ability to still open the files. If other consumer’s also support the extensibility mechanisms, then they too will have the ability to open files created based on future versions.

    -Brian

  9. Wouter Schut says:

    Brian,

    From the fine details you give on this site it is very clear that OpenXml is a direct derivative of Office 2007. This is a good thing if you want near 100% interoperability in a finite amount of time. But what I still do not understand is the politics and the marketing of this new format. For me and the EC this is the most important aspect of an open standard (from wikipedia):

    "The standard is adopted and will be maintained by a not-for-profit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.)."

    That is currently not the case with OpenXml because it is directly linked to the Microsoft Office implementation. Now many (if not all) remarks are dismissed because of Office 2007 compability. Shouldn’t there be a stable branche and a development branche for OpenXml which does allow major changes?

    I know ODF also inherits from OpenOffice.org but that format clearly went through a standards body which really changed a lot in the format. When can we expect OpenXml to evolve into something more vendor neutral?

    And could you please answer my questions I asked here: http://blogs.msdn.com/brian_jones/archive/2006/07/11/661843.aspx#665435

  10. Andrew Sayers had a great suggestion that I should have a page set up that gives an overview of the blog