Office Adopts New Windows Display Technology


Office 2013 has undergone a substantial shift to a relatively new display facility, Direct2D, and a new text facility, DirectWrite. These are the display facilities that are used on Windows Phone 8, the new Windows RT slates, and optionally on Windows 7 & 8. Up through Office 2010, the Windows Graphics Device Interface (GDI) and complex script support by Uniscribe were used. RichEdit 8.0 interfaces to both kinds of display technologies, but a given RichEdit instance uses only one of them. This post describes some of the advantages of using the new display facilities along with some of the problems we encountered in supporting them. One of the challenges was ensuring that math display wasn’t broken in the Direct2D/DirectWrite world. Another was to implement higher-level BiDi protocols using DirectWrite, which was designed to support the Unicode Bidi Algorithm (UBA) only.

Direct2D and DirectWrite are modern components that can take advantage of graphics acceleration hardware. DirectWrite supports better text placement using subpixel, device independent layout along with ClearType or other antialiasing methods. So in principle, it’s a big win to move forward to these components. Meanwhile GDI and Uniscribe have been around for years and have been taught to handle all sorts of nonobvious problems. It’s not easy to produce new code with their generality. Furthermore, RichEdit needs to be able to use either DirectWrite Version 1.0, which ships with Windows 7, or the more powerful DirectWrite Version 2.0 that ships with Windows 8. As a result, it was a bit of a tour de force to generalize RichEdit to interface smoothly to these three different display models. The factorization we’ve done has simplified getting RichEdit onto other platforms as well, such as the Android and iOS operating systems.

As an example of GDI tricks, if you specify the charset in creating a font object, GDI will ensure that you get a font that handles that charset. Admittedly charsets cover only a subset of the world’s languages (no Indic, Syriac, etc.), but they do cover many important languages. Another trick is if a character is an end-user-defined character (EUDC) in the Unicode Private Use Area, GDI will ensure that you see a glyph by searching through registered EUDC fonts. These characters are not defined in the Unicode Standard, so you can’t use them reliably for text interchange. But they are popular in CJK locales and a given machine may have fonts with the glyphs that the user wants.

Uniscribe was developed alongside LineServices to support both the UBA and rich-text higher-level protocols. This approach is important for RichEdit rich-text instances, since they use the keyboard and math higher-level BiDi protocols. In addition, digit substitution support requires breaking out numeric runs, which Uniscribe does. DirectWrite creates runs giving the same script value to alphabetic characters and numeric digits that are considered by context to be part of that script. The method we found to identify the digits in an Arabic run given by DirectWrite, for example, is that they are given a BiDi level of 2, whereas the strong RTL characters have a BiDi level of 1. We’ve dreamt up a number of other ways to ensure that Latin letters are rendered LTR (except in plain text with an RLO override embedding). And then you get little problems such as two Thai characters surrounding the Arabic comma U+060C. Since U+060C is considered common to several scripts, DirectWrite happily puts it in the Thai script run. But a Thai font cannot handle it. Font fallback is one way to see a glyph; RichEdit breaks the run and uses an Arabic font.

For the new immersive environment on slates and the Windows Phone 8, not only are GDI and Uniscribe absent, so are the functions handled by the venerable user.dll and OLE dll’s. The user.dll program library includes the OS SendMessage, MessageBox, CreateWindow, etc. In the immersive environment, RichEdit is only available in windowless mode. The client can still send RichEdit messages via the ITextServices::TxSendMessage() method. The advantage of dropping user.dll and the OLE dll’s is the relative simplicity of the model and the correspondingly reduced size. But doing so omits significant functionality, at least in the initial version. At the same time, the touch functionality is dramatically improved on Windows 8 and in the immersive environment.

OneNote 2013 is the first Office application that has a fully immersive version, thanks, in part, to the fact that RichEdit 8.0 can run in a fully immersive environment. OneNote 2013 has the math display facility and you can edit math using alt+= to insert math zones and the linear format to enter equations. One problem with OneNote 2013 is that for some zoom factors, horizontal lines are snapped vertically to the wrong pixel boundary. There wasn’t enough time to fix this problem for OneNote 2013, so you may notice some square roots with misaligned bars.

Another side effect of these huge changes is that we have had no time to improve the Office math editing and display facility for Office 2013. The main accomplishment math-wise is not to lose any math functionality, aside from misaligned bars in OneNote. Hopefully next time we can add some cool math features.


Comments (20)

  1. Royi says:

    It is really sad to hear there are no improvements in the math department.

    When you say next version, do you refer the official release or Office 14?

    Thanks.

  2. Nali says:

    No equation numbering in Office 2013…  

    Why am I not surprised??

  3. Dave Kellermann says:

    Wow, the year is 2012 and we still can't number an equation. Maybe I missed it, but the first thing I did after downloading the preview was to excitedly open Word and bring up the Equation Ribbon. No Dice.

    I dream that maybe my grandchildren will enjoy such a feature. Office 2047?

  4. davidacoder says:

    At this point I am really lost. Equation numbering is not some nice, optional feature. The lack of it makes the otherwise fantastic equation editor in Word pretty much useless for any semi-serious work. I am convinced that way more than half of all documents that have equations in them actually number them. If you type equations, most people have to number them.

    I did have some understanding that MS didn't manage to squeeze it in for the initial version. But not even TWO releases later? Six years later??

    Looking back, it seems the decision to include equations in OneNote was a big, big strategical mistake. Clearly by now the team is so busy with just keeping up support in various versions that there is no room left to make the thing useful in the app that arguably needs it most, Word. Do people use equations in OneNote a lot?

    It is really frustrating to see a team that produced such a wonderful piece of code be hold back by the typical MS tax of needing to integrate with things that no one needed. It would have been WAY better to first fix equations in Word (by giving them numbers) before adding them to all the other apps in Office. Major, major fail on the part of the program management of this feature.

    I should also give you my perspective: I teach at a large research university in the US. I have a lot of contact with a large and diverse group of people that uses equations. Every single person I talked to in the last couple of years said they aren't using the build in equation editor because it doesn't do numbers. This goes from students, grad students to professors.

  5. MurrayS3 says:

    I personally know how important equation number is, having coauthored over a 100 papers in theoretical physics and three books on laser physics. I pushed really hard to get equation numbering into Word 2007, 2010, and 2013 and hopefully it'll happen next time. I think people here think it's harder than it really is, what with most of the infrastructure already in place.

    By the way, in OneNote 2013 have you tried typing into a math zone something like

        E=mc^2#(3)<enter>

    ? You might be surprised!

    Many thanks for your feedback and that of many others who have commented or contacted me. It's comments like David's that convince people to adjust priorities.

  6. Royi says:

    I find Equations in Power Point and One Note equally important.

    MS has enough programmers to make it right in every application.

    I wish the did it in Office 2013.

  7. Nali says:

    If I understand it correctly, this version of Office 2013 is "only" the Consumer Preview (=Beta). Later, there should be the Release Preview (=Release Candidate) and then finaly the RTM version.

    So it still time for progress and improvements (for adding features). But unfortunately, it seems that the Office Word team has no effort to add new features (which are connected to text/math typography…).

    For comparison, the Internet Explorer team added many new features in each new Preview of Internet Explorer each 12 weeks!! The Internet Explorer team has shown the real progress of IE10 development and improvements. So, it is good reason to release the new version of Internet Explorer (IE10). And I will upgrade from IE9 to IE10.

    But what is the reason to release the Word 2013? Without new (typographical) features (and equation numbering!!) I do not need to upragde from Office 2010 to Office 2013.

    Sorry, but this is my opinion.

  8. MurrayS3 says:

    A good summary of Word 2013 enhancements is given in the Word blog post blogs.office.com/…/introducing-word-2013.aspx. One very cool thing is the great integration with the cloud. With it, you can access your documents from anywhere and collaborate better than ever before. Also the touch and Windows 8 integration is excellent.

  9. Nali says:

    I have known about the enhancements of Word 2013 desribed on the Word blog.

    Primarily I though that the main improvements of MS-Word (as a text and word processing application) should be in the area of typography. After Word 2007 introduced the new and advanced math editing and displaying feature, I believed that after similar improvements of ordinary text displaying and editing the MS-Word would be better for scientific text preparation than LaTeX. But it seems that the priority has changed…, changed to the cloud and so on. I work with sensitive data so I will not used cloud for saving the MS-Office documents.

    The math editing and displaying facility of Word shows the great potential of  this application in the field of math typography and this advanced philosophy should not be limited only to math.

  10. txt says:

    Microsoft announced this week that Office 2013 can open PDF files in Word and save them as Word documents (see [1]). Will math expressions in PDF docs be opened/saved as OMML expressions? Can you say more about it?

    [1] blogs.office.com/…/the-new-office-expands-file-format-options.aspx

  11. MurrayS3 says:

    PDF Reflow, the official name for the PDF to DOCX converter, currently leaves formulas as plain text or renders them as images. The PDF format usually does not contain information indicating when text is mathematical. All that is stored in PDF is absolutely positioned text, images, and graphics. Moreover sometimes PDF does not even contain the underlying Unicode values for mathematical text, so that when you copy text from the PDF, the result is messed up, although the original PDF looks correct.

    PDF Reflow tries to detect mathematical text and to render it with the correct semantics; the formula structure is preserved, but the text cannot be edited. When mathematical text is not recognized as such, PDF Reflow imports the same text that you get otherwise by copying the content from the PDF. A blog post was written recently on the Word blog related to PDF Reflow: blogs.office.com/…/unlock-pdfs-with-the-new-word.aspx.

    Ideally mathematical text in PDFs would be recognized and imported as OMML (Office MathML). Then you could edit it and use it in symbolic and numerical computations. Maybe that will happen some day. The team that made PDF Reflow belongs to the same organization that made the Windows Math Input Panel, which recognizes hand-written equations and converts them to MathML. Conceivably some of the Math Input Panel's logic could be applied to recognizing formulas in PDFs.

  12. Peter Hans-Lorenz says:

    Talking about Directwrite, which enables subpixel positioning and thus would make the "optimize character positioning for layout" setting much more bearable on high-DPI-but-not-yet-retina displays in Word, etc. if it weren't for it using greyscale anti-aliasing… why?! Please don't tell me it was just to make it the Office programs more Metro-I.

  13. Peter Hans-Lorenz says:

    OK, I have no correct myself, no greyscale AA in OneNote, but now why in Word, hopefully just a bug?

  14. Nali says:

    Finally, it seems that the Word 2013 now supports the new better line-breaking algorithm (for the ordinary text) based on the penalty principles.

  15. Nali says:

    Please, can I have a question?

    What is the purpose of CTRL+ALT+ENTER keyboard shortcut in MS-Word? What should it do?

    Thank you.

  16. MurrayS3 says:

    I don't think anything is associated with CTRL+ALT+ENTER. You can use it for your own purposes. Here's a list of Word hotkeys: allhotkeys.com/microsoft_word_hotkeys.html

  17. Jeff says:

    Given that OneNote is a RichEdit control why does it not support copying and pasting rtf?

    Thank you.

  18. MurrayS3 says:

    OneNote uses RichEdit for editing and display on a paragraph level. It doesn't use RichEdit for list numbering or tables. And it doesn't use RichEdit for file I/O since a file needs to deal with list numbering, tables, and other features of a OneNote page. So OneNote uses HTML and its own XML format. But it could in principle support RTF, especially at the paragraph level. I'll ping my OneNote friends.

  19. Kasper says:

    I always wondered why the text in Onenote 2013 looks so much better than Word 2013.

    That could explain it. Is there no way to turn directwrite on for Word 2013 ?

  20. Ron MVP says:

    Murry, thanks for the information.

    Sad that MS is focused on making 2013/Win 8 work on smartphones, but they ignored their bread and butter on the desktop …

    So MS was focused on equations in OneNote.  

    That must be the reason we still see file corruption due to misplaced XML tags associated with equations in Word 2013.  There are literally hundreds of people pleading for help on the http://answers.microsoft.com forum after having their documents trashed by this bug that has been around since 2007. Typically they are on the final edits and as a result are on a short dead line.

    Luckily for these users there are a couple of unpaid volunteers who hand edit the corrupt XML to recover most of the document content.

    All of this "wonderful" new technology is still commonly misconfigured  causing Office 2013 to display blurry fonts and ribbons (hardware acceleration is usually the culprit) … <sigh>