Building consensus around ODF 1.2


It’s been interesting to be a part of the ODF TC and learn about how an OASIS working group handles the maintenance work for an evolving standard. I’ve also been involved in INCITS V1, Ecma TC45, and more recently SC34 WG4, and each group has its own style, based on the governing rules, the personalities of the members and committee chairs, the nature of the work to be done, and other factors.


ODF 1.2 will be the next major deliverable from the ODF TC, and co-chairs Rob Weir (IBM) and Michael Brauer (Sun) have been leading discussions of various open proposals on the email reflector and in the weekly TC phone calls. Much progress has been made, and there are also many open issues remaining to be resolved.


One topic that has been debated extensively is the question of how to handle conformance in ODF 1.2. There is a new requirement in section 2.18 of the OASIS TC Process (http://www.oasis-open.org/committees/process.php#specQuality) which states:


A specification that is approved by the TC at the Public Review Draft, Committee Specification or OASIS Standard level must include a separate section, listing a set of numbered conformance clauses, to which any implementation of the specification must adhere in order to claim conformance to the specification (or any optional portion thereof).

The ODF TC has been discussing how the conformance clause of ODF might be modified for the 1.2 release to meet this requirement. This topic has been discussed in numerous weekly phone calls, and has become the most-discussed topic on the email reflector since I joined the ODF TC last year. There are currently over 200 emails on this topic in January and February of this year from 14 different TC members, including editor Patrick Durusau, well-known standards participants from South Africa (Bob Jolliffe), Brazil (Jomar Silva), Czech Republic (Jirka Kosek) and the US (Dennis Hamilton), ODF implementers (IBM, Sun, Microsoft, Novell, KOffice, Nokia, Adobe, Gnumeric), and others. Although the debate has been vigorous at times, it has all been conducted in a cooperative and respectful spirit of collaboration, and I’m sure I’m not the only person who has learned quite a bit from the discussion.


I’ll describe my perspective on the conformance debate in another post soon, but for today I just wanted to comment on the process we’re following, and how this debate is (hopefully) leading to consensus on this topic.


In December, Sun’s Michael Brauer (originator of the conformance proposal) revised his proposal to include one-level and two-level conformance alternatives. We then started discussing this in some detail in early January, as mentioned above.


The agenda for the January 19 TC call listed the conformance proposal as a discussion topic, and the minutes of that call state that “no consensus” was reached on the proposal. Then the agenda of the February 9 call included this item:


7. Discussion of Conformance Proposal, Version 8.
http://lists.oasis-open.org/archives/office/200902/msg00058.html
[Note: The Chair declares his intent to have a vote to approve this proposal during the meeting, if he perceives a consensus. Otherwise his intent is to request a 7-day electronic ballot on whether the proposal shall be approved]

The next call was this week’s call on Monday 2/16, and the agenda for that call didn’t mention the conformance proposal, although it does include an item for “Approval OpenDocument v1.2 draft 9 as Committee Draft.”


I didn’t attend this week’s call, as I’ve not attended several calls lately. Stephen Peront has been taking over responsibility for ODF and OIC TC engagement on our team, and he reported to me that the call included a vote on a new draft of ODF 1.2, which included the conformance proposal. Stephen felt he had to vote no on the committee draft, because it includes a conformance clause that we (and other TC members) have said isn’t yet ready.


Now I see that Rob Weir (IBM), co-chair of the ODF TC, has blogged about Monday’s approval of the latest committee draft, and he includes this paragraph in his post:


We also need to socialize and grow consensus around ODF 1.2, both from implementers, but also adopters and consumers of ODF. There is still work to be done here. For example, the TC vote on the Committee Draft 01 was not unanimous. We did not have the support of Microsoft or Novell. There are still disagreements over how we define conformance in the standard. We obviously need to continue discussing this topic. Since the final TC vote to request an OASIS Standard ballot requires 2/3 approval of TC members with no more than 25% disapproving, we’ll need a high level of consensus in the TC to move forward, including, hopefully, the support of Microsoft and Novell.

It’s not clear to me why Rob singles out Microsoft and Novell by name here. As you can see from the minutes of the meeting, Novell abstained, as did Dennis Hamilton. Dennis is not affiliated with a large vendor, but he is one of the most prolific contributors to ODF 1.2 in recent months.


As a person watching the ODF 1.2 process from a slight distance (reading the emails but not attending recent calls), it’s odd to see that the chair’s perception of a need to vote on the proposed conformance clause seems to have disappeared, and instead the clause was inserted into a committee draft at the last minute. It feels a bit like “pork” (to use an American slang political term), but I’d love to hear another explanation if there is one. The chair’s role is to determine consensus, so if the chair has determined that there is clear consensus on the conformance clause, that would be interesting to know. As it stands, my colleagues Stephen and Eric Patterson felt they couldn’t approve a draft that had a non-resolved issue inserted without clear consensus, and I agree with their decision. As stated in the minutes, “Some TC members that voted with “no” indicated that their “no” vote is due to conformance clauses which are included in the draft.”


Speaking of watching from a distance, it’s interesting to note how the large vendors have been conducting themselves in the gear-up toward final approval of ODF 1.2. OASIS rules allow each member to have a vote (as opposed to the rules in organizations like INCITS V1, where each company has one vote), so it’s possible for a large vendor to have multiple votes. For Microsoft, I’ve stopped attending the calls (and thereby lost my voting rights) while Stephen Peront has started attending (and gained voting rights), so we still have two voting members. I believe Sun has three voting members currently, and IBM is apparently up to four voting members, having added Don Harbison last week and Ma Yue this week.


I’ll revisit the conformance proposal soon, to explain our position in more detail. Meanwhile, if anyone can help me understand the process better, please let me know in the comments below. I remain hopeful that we’ll get an ODF 1.2 completed that will reflect the consensus of the TC.


Comments (22)

  1. Rob Weir says:

    Doug, I believe you missed the meeting of February 9th, but you should go back and read the minutes for it.  (You seem to lack a link for it in your blog post as well)  Conformance was discussed and consensus was reached.  I went around the room (virtually at least) and asked for everyone’s opinion, one by one, whether they could live with the proposal as written.  We spent quite a lot of time on this.  A few points were brought up and we made changes to accommodate those objections.  No one objected to the clause once these additional changes were included.  That’s why we directed Michael go forward to add the new language to the draft.  

    The minutes of that meeting were posted the same day, February 9th, and no one objected. The updated draft was posted by Patrick on the 11th with the note "The conformance clause has now been incorporated".  No one objected.  When the draft was discussed on 16th, Microsoft did not object then either.  In fact, the first indication that you were anything but happy was when we did the roll call vote and Eric and Stephen voted No without comment.

    I’m not going to make everyone happy.  That’s not part of the job description for a committee chair. I think it is an accomplishment to get 2/3 approval on a question which has sparked so much debate.  If you have a proposal that you think will lead to a higher degree of consensus, then go for it.  Send it out to the list and I’ll personally ensure that it gets on the agenda.

  2. Doug Mahugh says:

    Rob, I saw the 2/9 minutes (at http://lists.oasis-open.org/archives/office/200902/msg00123.html), but they just say this about conformance:

    * Conformance discussion

    TC members commented on what they found acceptable and unacceptable in the latest iteration of the draft conformance clause.

    There was consensus that two changes should be made to the current draft:

    1) Rename the “Host Language” conformance class to “Extended” conformance class.

    2) Remove the language around table:formula until that question can be separately discussed.

    Michael will make the above changes.

    There have been several dozen emails on conformance (and the related “Least Power” thread) since then, so it doesn’t look to me like there was consensus at that point in time.  Are you saying that there was consensus for approval of this clause after the two above changes were made?  Or just that there was consensus that these two specific changes needed be made?

    To use another political analogy, if we have consensus in the TC that Rod Blagojevich needs a haircut, that’s not the same as consensus that we will approve of him if he gets one.

  3. Mary McRae says:

    From my reading of the TC email list archives (http://lists.oasis-open.org/archives/office/) the conformance section is not yet complete. According to Michael’s email to the TC on 10 Feb, he states: "I have not added anything regarding the use of foreign elements and attributes in <*-properties> elements of conforming documents, since this should be discussed separately."

    There are frequently multiple iterations of a Committee Draft approved before the TC agrees to a 60-day Public Review, and oftentimes more Committee Drafts follow before the spec is approved as a Committee Specification. While a number of non-TC members have begun to review the spec, it has not yet entered its official review period. That’s still to come.

  4. Rob Weir says:

    Doug, there was sufficient support for the conformance clause, as amended, to move it into the draft.   The fact that the draft then received 2/3 approval demonstrates that my call on the level of consensus was accurate.  

    Is it considered then final and we’re unable to change it?  No.  We’ll continue to iterate, and if there is a proposal that has a greater degree of support than we’ll go with that.  

    The conformance clause is important and I would be disappointed if we did not have more spirited debate on the subject.  But I know it would help me immensely, and others on the TC as well, if you would articulate, on the TC list, exactly what you do not like in the current draft, and how you would make it acceptable.  Otherwise we risk just going around in circles.

  5. Jomar Silva says:

    Hi Doug,

    It is strange for me to read your text, because as Rob wrote, the conformance discussion reached a consensus (this was my impression too).

    Also, if you follow the "Least Power" thread you’ll find that at the end of it, nobody from Microsoft has commented anything… simply silence (and silence usually means agreement, doesn’t it ?).

    Anyway, it would be great to have you back on the TC, and in my opinion, the conformance clause now attend to most of Microsoft initial claims…

    Best,

    Jomar

  6. Doug Mahugh says:

    That’s interesting to hear your impression, Jomar.  I’ve heard from a few TC members (including non-Microsoft members) who have a different impression.  I wasn’t on the call as you say, so I don’t know firsthand what happened, but I’ve not seen an email to the TC mailing list stating that there is consensus on the conformance proposal.

    As for Microsoft’s silence in the recent threads, that’s simply because the conformance proposal has been evolving and we were waiting for a final form to evaluate, having already expressed our concerns about prior versions.  Stephen or I will provide very detailed information about our view of the current proposal shortly.

    By the way, did you see Mary’s observation above about Michael’s email on the  need to discuss foreign elements later?  Given that foreign elements are one of the most debated aspects of this proposal, and an area I’ve expressed opinions on (as you have), it’s hard for me to understand how we could have consensus without addressing that topic.

  7. hAl says:

    Is it correct to say that ODF produced by KOffice will no longer be conformant for ODF 1.2 because the extensibility currently used by KOffice is removed from ODF 1.2 ?

    So either KOffice drops the features responsible for the extensibility or it stays stuck on ODF 1.0 ?

  8. Doug Mahugh says:

    Yes, hAl that’s correct — any such extensions would not be allowed in a "Conforming OpenDoument Document" and would instead be called a "Conforming OpenDocument Extended Document."  The clear implication is that such documents are "less interoperable" than Conforming OpenDocument Documents, although as a practical matter I’ve seen numerous examples of interop problems between existing ODF implementations (such as the ones I blogged about recently) that have no such extensions involved.  This is at the heart of our objections to the approach of the conformance proposal — it solves a theoretical problem, and does nothing to address the real problems that users actually face today.

    to be clear, Office’s ODF 1.1 implementation in the upcoming SP2 doesn’t use foreign elements.  But this issue applies to implementations of ODF 1.0 or ODF 1.1 that extended ODF in any way during the years since publication of those standards.  One interesting area is spreadsheet formula syntax — since ODF has never had such a syntax in any published version of the spec, implementers have used other standardized formula languages such as the one in IS29500 (as is allowed in the current ODF spec).  Is it the intent of the current conformance approach to make all such documents no longer conforming ODF documents?  And is such a move in the best interest of users of the standard?

    As I’ve said, we’ll be explaining our position in detail in the next few days, both to the TC and more broadly.  It’s worth noting that this blog post is not about conformance per se, but rather about the Rob’s blog post, which calls out Microsoft and Novell as if we’re obstructing consensus (and ignores the most prolific unpaid contributor to ODF 1.2, Dennis Hamilton).  I think the TC needs to be run in a way that truly strives for consensus, and narrowing the issue to a list of major vendors/competitors and their positions didn’t strike me as a useful approach.

  9. hAl says:

    Could someone explain the difference between shall and should from the conformance clause releated to this issue:

    It shall be able to parse and interpret OpenDocument documents  …

    It should be able to parse and interpret conforming OpenDocument extended documents …

  10. Doug,

    you say

    "One interesting area is spreadsheet formula syntax — since ODF has never had such a syntax in any published version of the spec, implementers have used other standardized formula languages such as the one in IS29500 (as is allowed in the current ODF spec).  Is it the intent of the current conformance approach to make all such documents no longer conforming ODF documents?  And is such a move in the best interest of users of the standard?"

    Isn’t it a bit far stretched? Changing the conformance clause of ODF 1.2 will not affect versions of ODF prior to ODF 1.2 and as such documents claiming conformance to ODF 1.0 or ODF 1.1 will indeed be entitled to the "OpenDocument 1.0/1.1 conformant"-sticker.

    However, if OpenFormula is not finished before publication of ODF 1.2, the consequences you mention here will be correct. An OpenDocument spreadsheet containing formulas will not be conformant to the "pure ODF 1.2" but only to "Extended ODF 1.2". THe same goes for OpenDocuments containing MathML, as I wrote to the office-comment mail list a few days ago.

    i think that having a conformance clause excluding spreadsheets with formulas and text documents with math content (quite normal document artifacts, if you ask me) is a bit peculiar and I hope that ODF TC changes its mind on this before ODF 1.2 is voted on in OASIS.

  11. Doug Mahugh says:

    @hAl "shall" is a normative requirement, so those clauses are saying, in essence, that a conforming consumer is allowed to refuse to load extended documents.

    Jesper, if I understand your point correctly, you’re saying that you don’t mind if documents containing IS29500 formula markup are unable to ever be conformant to "pure ODF 1.2" but you do think it would be a bad thing if documents containing OpenFormula markup can’t claim conformance to "pure ODF 1.2."  Is that correct?

    My view is that this would needlessly punish users of ODF who currently save formulas in the only ISO-standard formula markup language available for this purpose.  We support OpenFormula and continue to contribute to its development, but I think the standards process should not be used to attempt to force users toward a particular option, especially an option that will likely appear as an ISO standard years after the first option was available.

  12. orcmid says:

    I’m beginning to feel embarassed by how often my name (Dennis Hamilton) is appearing in this post and its comments.  Although I have been very active in recent discussions, especially around maintenance and continuity (up- and down-level compatibility) issues, of which conformance is a part for me, that volume should not be taken as a measure of my contribution to the ODF 1.2 Committee Draft 01, which is somewhere between miniscule (language around the office:version attribute) and non-existent.

    With regard to my abstention, I chose not to offer any explanation when Rob asked whether anyone who didn’t vote yes wanted to say anything about their choice.  There is no requirement to explain a voite and I shall leave it at that.

  13. Doug,

    I want to have a formula language specified in ODF 1.2 in much the same way it is defined in IS29500 (18.17). As a principal matter, I really don’t give a damn if it is one (IS92500:2008 Part 1, 18.17) or another (OpenFormula).

    I want ODF to say: "When doing formulas in spreadsheets, use this notion" and NOT "when doing formulas in spreadsheets, do whatever you want".

    The public draft of ODF 1.2 states on page 2:

    "Part 2 defines a formula language to be used in OpenDocument documents."

    So if ODF 1.2 will indeed use OpenFormula as the content of Part 2 and constrain usage of formulas to this, then "yes", I don’t ever want an ODF documents containing IS29500-formulas to be "pure ODF 1.2".

    If OOO will indeed be done, the namespace "urn:oasis:names:tc:opendocument:xmlns:openformula:1.0" should be added to the list of known namespaces in ODF 1.2 in section 1.3.

    The attribute "formula" on the <cell>-element has currently this description:

    "Every formula should begin with a namespace prefix specifying the syntax and semantics used within the formula."

    This indicates to me that there are no intentions to require OpenFormula to be the first-class citizen of ODF for specifying formulas.

    Can you verify this? Can you tell us if OpenFormula will be done in time for ODF 1.2 approval in OASIS and if it will be /the/ way of specifying formulas?

    I think it will be really, really sad if ODF – here after almost 4 years since approval in OASIS – will not have a specified formula language.

  14. Doug,

    "(…) this would needlessly punish users of ODF who currently save formulas in the only ISO-standard formula markup language available for this purpose."

    Seriously, Doug … please don’t play the "reuse exisiting ISO-standards or users will be punished"-card.

    :o)

  15. hAl says:

    Doug, I still do not under stand what "should" means compared to "shall".

    If consuming odf extended files is optional than O would think "may" should be appropriate.

  16. Alex Brown says:

    @hAl

    The ISO/IEC Directives Part 2 Annex H (which defines these usages), provides equivalent forms which help to explain these meanings. Broadly:

    shall = has to

    should = ought to ("it is recommended that")

    may = is allowed to

    can = is able to

  17. AndréR says:

    Moinmoin Doug,

    in the references.odf document posted by Rob Weir I read

    "[OLE]     Kraig Brockschmidt, Inside OLE, Microsoft Press, 1995, ISBN: 1-55615-843-2

    This is a good book, but out of print. There must be some official Microsoft specification for OLE 2 posted on the web someplace.  We should reference that."

    Could you clarify the situation concerning available uptodate (online)  documentation for OLE2 technology that could be easily referenced?

  18. Doug Mahugh says:

    Andre, here’s a link to the latest OLE documentation online: http://msdn.microsoft.com/en-us/library/cc313062.aspx

  19. hAl says:

    Thanks doug. That explains it better.

    I would prefer standards using terminology like

    "it is recommended that"

    which is easier to interprete in a world where not everybody is a native english speaker but many are likely to encounter standard text in English whem inplementing it

  20. hAl,

    I don’t agree with you here. It is important when reading/writing a standard that the vocabulary is consistant.

    Remember that a standard is essentially a set of rules to follow when implementing it. I actually like that the wording is consistant across standards.

    Everyone making guides and "how-tos" are more than welcome to use more "everyday language", but I think the wording in standards should remain the same.

    :o)

  21. hAl says:

    @Jesper

    I think it is poor wording because I now get the impression that shall will often beinterpreted as "not required" and as such has no more formal conformance requirements than "may" or "can"

    I do not consider that to be very consistant already.

  22. hAl says:

    I will also direct you to a dictionary

    http://www.merriam-webster.com/dictionary/shall

    Where you can see that "shall" is multi interpretable english that can refer to mandatory or must requirements

    I think it is very bad to use that kind of interpretable language in standards especially in a world where more an more english written standards are used  by non-english speakers.