Massachusetts ETRM V4.0 recommends Open XML and ODF 1.1

A month ago, the Commonwealth of Massachusetts posted a draft of its revised Enterprise Technical Reference Model (ETRM) policy, requesting comments from the public. As I noted in a post on June 30, "The ETRM (Enterprise Technical Reference Model) provides an architectural framework used to identify the standards, specifications and technologies recommended for use in the Commonwealth’s computing environment. This update included two document-format changes: updating the OpenDocument Format specification to version 1.1, and adding Ecma 376 (Office Open XML)."

The public-comment period ended on July 20, and the revised ETRM policy (version 4.0) is now posted on the ITD (Information Technology Division) section of Massachusetts's web site. Both of the proposed changes have been included: ODF 1.1 and Ecma 376 (Open XML) are now recommended formats in Massachusetts.

The ETRM and XML

The ETRM, as its name implies covers a variety of enterprise technology areas. Document formats are covered in the "Information Domain" section of the ETRM.

At the highest level, the ETRM specifies XML as the core technology to enable data interoperability. After noting that "XML has become the lingua franca of application integration, facilitating application interoperability," the ETRM makes the following recommendation:

"Agencies should consider the use of XML for all projects, and should implement XML, unless there are compelling business reasons not to do so. XML should always be considered when undertaking new work or when beginning a major overhaul of an existing system."

They go on to describe the importance of XML Schema, XPath, XSL, and XQuery, and how those technologies can support an XML-based approach. This perspective isn't unique to governmental agencies, of course. All organizations are moving to XML-based formats, and you could change "agencies" to "departments" in that statement and it could be a quote from an internal memo in any major technology-driven organization, whether public or private, consumer or vendor. Open and standardized XML formats are fundamental to interoperability in today's environment, and for the foreseeable future.

Document formats recommended by ETRM V4.0

The ETRM doesn't just recommend XML-based formats, however. It's a very pragmatic document that looks at the realities of current technology and trends, and it recommends six different document formats, each with well-defined areas of application. That list includes four formats in the "Open Formats" section (ODF, Open XML, plain text and HTML) and two more under "Other Acceptable Formats" (PDF and RTF).

The description and guidelines for the first two open formats, ODF 1.1 and Ecma 376 (Open XML), are nearly identical. Each is described as a "standardized XML-based file format specification suitable for office applications" that "covers the features required by text, spreadsheets, charts, and graphical documents." Each format also includes the following migration guidance: "All agencies are expected to migrate away from proprietary, binary office document formats to open, XML-based office document formats."

Where the descriptions of the two formats differ most is in the list of supporting applications. For ODF, the ETRM says "The OpenDocument format is currently supported by a variety of office applications including OpenOffice.org, StarOffice, KOffice, NeoOffice 2.1, and IBM Workplace. In addition, there are a number of translator software solutions that enable other office suites, such as Microsoft Office, to translate documents to and from OpenDocument Format for text documents. In the future, there will be translator software solutions for spreadsheets and presentations as well."

For Open XML, the ETRM says "The Open XML format is currently supported by a variety of office applications including Microsoft Office 2007, OpenOffice Novell Edition, and NeoOffice 2.1. Corel has announced Open XML support for WordPerfect 2007. In addition, the Microsoft Office Compatibility Pack enables older versions of Microsoft Office such as Office 2003, XP and 2000, to translate documents to and from Open XML Format for text, presentation and spreadsheet documents."

Continuing with the "Open Format" options, two other formats are recommended: plain text and HTML. For plain text, the ETRM notes that it "is the most portable format because it is supported by nearly every application on every machine," but it "should not be used for documents where formatting is important or is part of the official record." The ETRM describes HTML as "the preferred format for documents that will be accessed through the Internet/Intranet or using a web browser."

Rounding out the list is two non-open formats, PDF and RTF. PDF "may be used for documents whose content and structure will not undergo further modifications and need to be preserved," and RTF may be used "for ease of interoperability among different systems however XML-based document formats must be considered as a first choice."

Statement on the comments received

The ETRM update was accompanied today by a Statement on ETRM v4.0 Public Review Comments from Henry Dormitzer, Undersecretary of Administration and Finance, and Bethann Pepoli, Acting CIO of the Commonwealth of Massachusetts.

I'd recommend reading that statement carefully, if you'd like to understand how Massachusetts reached this decision. It's clear that they've been lobbied pretty hard by vendors, lawyers, and others with interests in document-format legislation, and there's a bit of defiance in the tone of certain comments:

  • "The Commonwealth continues on its path toward open, XML-based document formats without reflecting a vendor or commercial bias in ETRM v4.0."
  • "[We] believe that the impact of any legitimate concerns raised about either standard is outweighed substantially by the benefits of moving toward open, XML-based document format standards."
  • "Massachusetts is the first state to adopt a policy encouraging open, XML-based document formats."

Other reactions

My reaction is simple: I believe this is a great development in Massachusetts, a good thing for both citizens and vendors. It continues the trend we're seeing in Europe (Denmark, Switzerland), and I expect it's the first of many similar decisions we'll see in US state governments going forward. Other states have looked at the cost of mandating single-format lock-in for government agencies, and will likely reach similar conclusions about the value of choice.

Melanie Wyne, Executive Director of ISC (Intiative for Software Choice), had this to say about the ETRM announcement today:

“Having objective, performance-based policies harnesses the best the industry has to offer. Ideally, such policies should embrace a definition of interoperability that is focused on the goal— i.e., sharing data across diverse systems—rather than on the means to obtain the goal. Specific technical mandates defeat this goal."

“Office applications and their formats exist within a dynamic environment. As such, standards that relate to these formats must also be permitted to evolve and retain the flexibility so critical to today’s IT marketplace."

“The ISC is pleased to see that the ETRM seeks to promote choice, relying on the dynamic and evolutionary nature of the IT industry – not inflexible technical mandates – to better serve agencies and constituents.”

Not everyone agrees that the revised ETRM goes far enough. For example, Jonathan Zuck, President of ACT (Association for Competitive Technologies), issued a press release today in which he said:

“The Massachusetts open format policy has evolved into a much more effective solution to the challenges of interoperability, competition, and long-term document access, but it can still be improved. The Commonwealth has moved beyond strict technology mandates and has developed a more flexible framework for achieving its goals. It allows for agencies to choose from multiple ‘open standards’ to meet individual goals, and even seems to allow agencies to select non-open standards under certain circumstances.”

“The one real limitation to the policy is the rigid definition of ‘open standard’ used in the ETRM. The policy limits the Commonwealth’s choices to ’open standards,’ when the goals could be achieved with merely ‘open formats.’ While small firms are often willing to open up their formats and technologies, they often do not have the political clout to move their formats through an open standards body the way IBM, Sun, and Microsoft have done. In fact, even open source formats like Ogg Vorbis would be locked out. Yet, these small firm technologies may better meet the needs of the Commonwealth and individual agencies.”

“In addition, by committing only to broad open standards approved by international committees, the needs of smaller user groups can be overlooked. As the ETRM acknowledges, there are currently no office applications with native ODF support that provide accessibility for persons who use assistive technology devices.”

Stay tuned, there's surely more discussion to come in this area. As it says on the ETRM home page, "the ETRM continues to be a work in progress."