First here’s how these features are revealed to sighted users. In all math-enabled Office apps, the innermost math argument containing the IP is lightly shaded and selected text has the same selection background color as text not in math zones. In PowerPoint and OneNote, the math object containing the IP is shaded a bit more lightly than the argument and if the IP isn’t in a math object, the whole math zone has this lighter shading. In Word, the math zone is enclosed in a boundary and the object containing the IP doesn’t have the lighter shading. The user always knows what kind of an argument is involved just by looking at the built-up (Professional) display. This information is also conveyed in math fine-grained speech.

A refreshable braille display typically has a row of 40 or 80 8-dot cells with the dots represented by small rounded pins that are raised by solenoids. The dots are arranged in two columns of four dots. The left column is numbered starting at the top 1, 2, 3, 7 and the right column is numbered starting from the top 4, 5, 6, 8. Like most braille codes, the Nemeth math code uses the dots 1 through 6. This leaves dots 7 and 8 for UI purposes, although dot 7 is occasionally used to indicate upper case. The Nemeth code precedes a letter with the capitalization indicator “⠠” (lone dot 6) to get upper-case letters, e.g., “⠠⠁” for “A” since “⠁” is the braille code for the letter a. So, we don’t use dot 7 to indicate upper case, at least in math zones.

The regular math braille display shows the whole math zone in braille, limited only by the number of display cells. This gives a lot of context to math braille, significantly more than math speech provides, but not as much as screen or paper.

Typically, selected text appears with both dots 7 and 8. So if “a” is selected, it appears as “⣁”. This approach seems well suited to math expressions as well.

We’re left with needing ways to identify a math zone and the insertion point and to highlight the innermost argument containing the IP if any. Braille displays don’t have multiple shading levels, only two extra dots! They also have hot keys.

The IP needs a cell by itself to stand out. As described in the post Text Insertion Point, the IP is *in between* two characters in rich text, although for plain text one can get away with thinking of the IP as being on top of the character that actually follows the IP. Built-up (Professional) math text is rich text notably because it has special display constructs, such as stacked fractions, multilevel subscripts and superscripts, integrals, matrices, etc. For this purpose (and perhaps others), dots 7-8 “⣀” comprise a simple, effective IP. Admittedly this is the same as a lone selected space, but it seems to be readily distinguishable since the user usually knows when something is selected versus having an IP and s/he can easily move the IP (or hit the IP-identification hot key coming up) to check if in doubt.

To reveal the innermost argument containing the IP, one can turn on dot 8 for the characters in that argument. This is similar to the argument shading used in regular displays. To illustrate this approach, consider the fraction 1/2π, which in built-up form is given by the Nemeth braille string “⠹⠂⠌⠆⠨⠏⠼”. If the IP precedes the 2 in the denominator, the braille display would have “⠹⠂⠌⣀⢆⢨⢏⠼”.

At first the dot 8 in the denominator cells here might be confusing, but it resolves ambiguities as to whether the IP “⣀” is inside or outside of a math object. This isn’t a serious problem with fractions since the fraction start, fraction bar, and fraction end appear as the explicit braille codes ⠹,⠌,⠼, respectively, although it’s always helpful to know when the IP is in a math argument. But consider the quantity *a*², which is given in Nemeth braille by “⠁⠃⠆”. In Office apps and MathML, superscripts are represented by two arguments, the base and the superscript. If the IP precedes the base, is the IP at the start of the base or at the start of the superscript object? That position is ambiguous without the dot 8 option. With dot 8, you can tell the difference: in “⣀⠁⠃⠆” the IP precedes the superscript object, while in “⣀⢁⠃⠆” the IP is inside the base in front of the “a”. Distinguishing these positions is essential for unambiguous editing of mathematical text.

Dot-8 highlighting reveals when the IP is at the start or end of an argument or somewhere in between. But it doesn’t define what kind of argument. To get this kind of information on a braille display, it’s handy to have an IP-identification hot key that flashes the name of the argument containing the IP (or “math zone” if the IP isn’t inside an argument) onto the braille display. This name needs to be localized to the current user language, while the regular braille for the math zone is globalized by nature. For example in English, depending on where the IP is in a denominator, the hot key displays “start of denominator” (⠎⠞⠁⠗⠞⠀⠕⠋⠀⠙⠑⠝⠕⠍⠊⠝⠁⠞⠕⠗), “end of denominator” or just “denominator”. This is more informative than the corresponding math speech, which only announces the kind of argument when the IP is at the end of an argument, or the kind of math object when the IP is at the start of an object. This difference occurs because fine-grained speech needs to say the character at the IP, whereas the math braille display continuously shows the characters around the IP, limited only by the number of display cells.

It might be worth having options to enable/disable dot-8 highlighting according to user preference. Even without the dot-8 highlighting, the user can resolve ambiguities by hitting the IP-identification hot key so some users might prefer to work with the simpler braille display.

Lastly, how do you reveal a math zone? If the IP is inside a math-object argument, the presence of dot 8 is a good indicator. As described in the post Braille for Math Zones, math zones start with “⠸⠩” and end with “⠸⠱”. So, the start and end of a math zone are not ambiguous in math braille. In the Microsoft Office math representation, whether the IP at the start of a math zone is inside the math zone or outside is revealed by shading or the Word math-zone border, since the character position is the same for both cases. Ditto for the end of a math zone. I tried setting dot 8 for all cells in a math zone when the IP is in a math zone, but not inside an argument, but it seems too messy. So hopefully the math zone start and end delimiters will suffice; the user can hit the IP-identification hot key to find out whether the IP is in a math zone.

With these uses of dots 7 and 8 and the IP-identification hot key, you can edit virtually all levels of mathematics using a refreshable braille display in an interoperable way with sighted users. Pretty cool, eh?!

]]>We compare STIX 2 Math (STIX2Math.otf) and Cambria Math in an equation and in an expression with nested parentheses. To make the equation accessible, here’s the Nemeth braille and UnicodeMath for it with math-italic variables mapped to upright for simplicity

⠹⠂⠌⠆⠨⠏⠼ |
⠮⠰⠴⠘⠆⠨⠏⠐⠹⠙⠨⠹⠌⠁⠬⠃⠀⠎⠊⠝⠀⠨⠹⠼ | ⠀⠨⠅⠀ | ⠹⠂⠌⠜⠁⠘⠆⠐⠤⠃⠘⠆⠐⠻⠼ |

1/2π | ∫_0^2π dθ/(a+b sin θ) | = |
1/√(a^2-b^2 ) |

The equation occurs, for example, in mode-locking phenomena in lasers, clocks, tuning forks, and other sustained oscillators. Hence the name “mode-locking equation”. More on that at the end. Here’s the equation in STIX Math 2.0

And here it is in Cambria Math

Many symbols look very similar, but note that the STIX *θ* has wider stems on top and bottom while Cambria Math has wider stems on left and right. The images are snipped from a Word document.

Similarly, here’s the nested parenthesized expression ((((((((a+b)))))))) in STIX Math 2.0

And in Cambria Math

It’s interesting to see how much larger the parentheses become in Cambria Math. The parameters guiding the layout of both are given in their respective OpenType math tables.

Re mode locking, Huygens is probably best known for his wave theory of light, but he was also the one who first put the pendulum into a clock. One day he observed that two pendulum clocks which ticked at slightly different rates when apart, ticked at the same rate when hung close together on the same wall. This mode locking phenomenon can be described, in part, by the equation above (see p. 52 of *Laser Physics*, Sargent, Scully and Lamb, Addison-Wesley, 1974)

This latest version of UTR #25 includes a discussion of glyph variants for the digit 0 and for the empty-set symbol ∅. Specifically, it notes in Section 2.7 that sometimes, specific glyph forms are chosen by notational style or are needed for contrast with other notation in the same document. For example, the symbol U+2205 ∅ EMPTY SET can be found in its slashed zero-shaped glyph form in documents typeset in TeX that use the command \emptyset, or in contexts where it is contrasted with the semantically distinct slashed digit zero.

For this and certain other well-established glyph variants of mathematical symbols, standardized variation sequences were added to the Unicode Standard. Thus, for example, the standardized variation sequence <U+2205, U+FE00> represents the oval variant of the empty-set symbol. To avoid the misuse of that sequence for the glyph variant of the digit zero with a short diagonal stroke “0”, the standardized variation sequence <U+0030, U+FE00> represents the digit zero glyph variant.

The report also uses the name UnicodeMath to refer to the math “linear format” in Unicode Technical Note #28. Since there are several math linear formats, it’s helpful to use a more precise name for that linear format.

]]>The most obvious simplification of relegating math to math zones is that it nearly eliminates the need for the Nemeth braille numeric indicator ⠼ in math zones since contractions aren’t used in math zones. In fact, the only need for the numeric indicator in a math zone is in the very rare case that the math style changes inside a number as in **12**34. The code ⠼ is also used to end fractions, but that use never was ambiguous since contractions aren’t allowed in fractions and Nemeth digit codes represent digits inside fractions.

One common scenario for which ⠼ can be omitted is when a digit follows a space in a math zone. Relational operators like = and → are always surrounded by spaces in the Nemeth standard. For example, the limit

is officially written in Nemeth braille as

⠐⠇⠊⠍⠩⠭⠀⠫⠒⠒⠕⠀⠼⠴⠻⠀⠹⠎⠊⠝⠀⠭⠌⠭⠼⠀⠨⠅⠀⠼⠂

Here, the Nemeth standard specifies that since the 0 (⠴) and 1 (⠂) follow a space, they should be preceded by a ⠼. But in a math zone no ambiguity exists if the ⠼ is omitted, resulting in more efficient input and simplified braille display.

The numeric indicator must precede all numbers for UEB math braille in math zones and elsewhere, since UEB uses the same codes (⠁⠃⠉⠙⠑⠋⠛⠓⠊⠚) for 1-9, 0 as for the letters a through j, respectively. The Nemeth digits use those codes shifted down a row (⠂⠆⠒⠲⠢⠖⠶⠦⠔⠴) which don’t overlap letters or other symbols. The Nemeth digit codes appear as parts of some mathematical symbols, such as ⠫⠲ for □. This choice is mnemonic since a square has 4 (⠲) sides. No ambiguity there either. Some digit codes are used in common punctuation, such as 4 (⠲) in ⠸⠲ for period (as distinguished from decimal point). This use isn’t ambiguous since the ⠲ is preceded by the punctuation indicator⠸.

In conclusion, it seems that the most efficient braille for math zones is Nemeth math braille minus the use of the numeric indicator except in the rare switching of numeric styles as in *ab**cd*. Such math braille is international except where natural language words are inserted such as “if”. In contrast, the text surrounding math zones is in a specific natural language. UEB is the obvious choice for English text. When Nemeth math zones are embedded in UEB, they start with ⠸⠩ and end with ⠸⠱ in accord with Using the Nemeth Code within UEB contexts. Other languages may be able to embed Nemeth math zones with these codes too. Math braille isn’t simple, but in modern documents it’s simpler than you may have thought ☺.

The present post describes how MathML could be used in generating fine-grained speech. The trick is to reveal where the insertion point (IP) is so that the user knows where the next character input will go.

To see how this works, consider the fraction 1/2π displayed in built-up form as

The coarse-grained speech for this (in English) is “1 over 2 pi”. The fine-grained speech resulting from moving right one character at a time is

“start fraction”

”1”

“end numerator”

“2”

“pi”

“end denominator”

With character navigation, Narrator speaks these strings for the fraction in Word, PowerPoint, and OneNote documents. Hearing this speech, the user knows where the IP is and hence where the next character typed is entered. To enable editing, MathML content needs to offer the same functionality.

The MathML for the fraction is

<mml:math xmlns:mml=”http://www.w3.org/1998/Math/MathML” display=”block”>

<mml:mfrac>

<mml:mn>1</mml:mn>

<mml:mrow>

<mml:mn>2</mml:mn>

<mml:mi>𝜋</mml:mi>

</mml:mrow>

</mml:mfrac>

</mml:math>

This doesn’t name the numerator and denominator explicitly. Instead, the numerator is defined to be the first child of the <mml:mfrac> entity and the denominator is defined to be the second. The MathML can be used in generating the speech “1 over 2 pi” in a natural language, that is, the coarse-grained speech. Fine-grained speech needs MathML that identifies what’s at the insertion point, which can be a character, the start of the fraction, the end of the numerator, or the end of the denominator. The MathML above doesn’t offer such information.

There are at least two ways to produce such per-character-position speech using MathML by including an <maction> entity. For the first way, when the IP moves by character in front of the fraction, the MathML would be

<mml:math xmlns:mml=”http://www.w3.org/1998/Math/MathML”>

<mml:maction actiontype=”input”>start fraction</mml:maction>

</mml:math>

Dropping the <mml:math> entity for brevity, the MathML output for subsequent move-by-character navigation actions would be

<mml:maction actiontype=”input”>1</mml:maction>

<mml:maction actiontype=”input”>end numerator</mml:maction>

<mml:maction actiontype=”input”>2</mml:maction>

<mml:maction actiontype=”input”>pi</mml:maction>

<mml:maction actiontype=”input”>end denominator</mml:maction>

The text in the <maction> entity can be localized into various languages. If this approach becomes popular, it’d be worth standardizing on text strings like “end numerator” to help users as well as localization. The Microsoft Office math-speech engine produces strings with 16-bit speech tokens that index sets of language strings in over 18 languages. But that process occurs internally. For general implementation by ATs, it seems better to use a set of standardized English strings that an AT can associate with other language string sets. A set of such English strings can be obtained by running Narrator over a Word document with equations on an English operating system.

A second way to produce such per-character-position speech using MathML is to generate the MathML for the math object that has the insertion point and include an <maction> revealing where the IP is. For example, if the IP is at the end of the numerator in the fraction above, the MathML would be

<mml:math xmlns:mml=”http://www.w3.org/1998/Math/MathML” display=”block”>

<mml:mfrac>

<mml:mrow>

<mml:mn>1</mml:mn>

<mml:maction actiontype=”insertion point”/>

</mml:mrow>

<mml:mrow>

<mml:mn>2</mml:mn>

<mml:mi>𝜋</mml:mi>

</mml:mrow>

</mml:mfrac>

</mml:math>

Since this MathML has the full context of the insertion point, the AT can create suitable speech. It requires more analysis by the AT than the first <maction> approach, but is more flexible. Such approaches using <maction> are quite general and don’t need specialized methods to decode the math in memory. They could work for all operating systems and applications that support MathML.

(Thanks to Sue-Ann Ma, Neil Soiffer, James Teh, Volker Sorge, Peter Frem and Ziad Khalidi for encouraging me to come up with a way to use MathML for editing).

]]>Coarse-grained speech isn’t tightly synchronized with the characters in memory and cannot be used directly for editing. It’s relatively independent of the memory math model. In contrast, fine-grained speech is tightly synchronized with the characters in memory and is ideal for editing. It depends on the built-up math model (“Presentation Math”), which is the same for all Microsoft math-aware products but may differ from the models of other math products. Coarse grained navigation between siblings for a given math nesting level can be done with Ctrl+→ and Ctrl+← or Braille equivalents, while fine-grained navigation is done with → and ← or equivalents. The latter allows the user to traverse every character in a math zone. Two special cases are 1) when the IP is directly before the math zone being queried by UIA and 2) when the IP is still in the range’s math zone, but at the end. For 1) the user needs to know that typing something won’t be in the math zone. Typing → then puts the IP into the math zone and typing enters characters inside the math zone. And for 2), the user needs to know that the IP is at the end of the math zone and still in the math zone. Case 1) returns “equation” followed by the speech for the math zone. Case 2 returns “end equation”. (Since many math zones aren’t equations, this choice of words might be a little misleading sometimes, but hopefully not too much so).

The languages with math speech support include Danish (da-DK), German (de-DE), English (en-US), Spanish (es-ES), Finnish (fi-FI), French (fr-FR), Italian (it-IT), Japanese (ja-JP), Korean (ko-KR), Norwegian (nb-NO), Dutch (nl-NL), Polish (pl-PL), Brazil Portuguese (pt-BR), Portugal Portuguese (pt-PT), Russian (ru-RU), Swedish (sv-SE), Turkish (tr-TR), PRC Chinese (zh-CN), Taiwan Chinese (zh-TW).

Math speech is produced by “building down to speech”, sharing the code and concepts of building down “Presentation Math” to UnicodeMath. This approach creates math speech just as fast as it creates UnicodeMath and is faster than representing math zones in other math formats like MathML. A string of language tokens is created and then converted to the active natural language.

On a technical level, math speech is implemented in the RichEdit dll (Office’s riched20.dll) by the GetMathSpeechText function, which has the prototype

HRESULT GetMathSpeechText (ITextRange2 *prg, BSTR *pbstr, LONG Flags)

Coarse-grained math speech is returned in *pbstr if the range prg selects more than one character while fine-grained speech is returned if prg references an insertion point or selects only one character. GetMathSpeechText() uses the same subset of ITextRange2 methods used by MathBuildDown() and hence can be used by all Microsoft Office math-aware applications on all major platforms (Windows, iOS, Mac, and Android). Key methods include ITextRange2::GetChar2() to fetch individual characters from memory and ITextRange2::GetInlineObject() to find out what kinds of math objects are in memory.

Math speech is exposed to UI Automation clients via methods of the UIA interface ITextRangeProvider. So, in principle any AT that uses these methods automatically gets math speech for math zones. Nevertheless, it’s desirable for AT’s to know if math zones are involved. One approach is to identify math zones by a new, explicit UIA math-zone object or by a custom object with a localized name like “math zone”. But a more efficient approach that mirrors what’s in memory is to have a math-zone format attribute. Specifically, TextUnit_Format is one of the units supported by ITextRangeProvider::ExpandToEnclosingUnit and ITextRangeProvider::MoveEndpointByUnit. To find out an attribute, such as UIA_IsItalicAttributeId, of a TextUnit_Format instance, a client calls ITextRangeProvider::GetAttributeValue. AT’s could know if a math zone is active if a new attribute ID, UIA_IsMathZoneAttributeId, is added to identify math zones.

]]>The focus here is on math zones, which are text ranges that have math typography, rather than normal typography. Natural language contractions are not used in math zones, so hopefully we can get general globalized math-symbol mappings. When math zones are embedded in UEB, a math-zone start delimiter would be ⠸⠩ and the math-zone end delimiter would be ⠸⠱ in accord with Using the Nemeth Code within UEB contexts. Math zones are key to working with technical documents since math-zone typography and conventions differ from those for normal text. So, a user needs to know when a math zone starts and ends.

The Nemeth specification describes several symbol construction techniques. Some very productive ones are illustrated in the following table, which also includes the section number in the Nemeth specification. The structure codes such as the termination indicator ⠻ are displayed in red

The mappings given in the table below resulted from scouring the Nemeth specification, which is a pdf file of scanned images. As such it offers no search or link capabilities, and a paper version is more useful than the electronic version. It’s the first document I’ve printed in years, other than occasional tickets, boarding passes, and legal authorizations. There’s also a nicely formatted Nemeth specification in French complete with a navigation pane with links to all the rules. You can search the text including finding braille sequences, since the sequences are encoded in Nemeth Ascii braille. This is valuable in learning about sequences in general and whether a potentially new sequence is already defined. The combination ⠠⠱ doesn’t appear as math in the French specification except as part of the extended tilde ⠈⠠⠱, so that seems like a good candidate for encoding the missing reversed tilde ∽ (∼ is given by ⠈⠱). The French version’s content has differences from the original English version, so I checked both in creating the table entries. It would be nice if someone would enter the 1972 Nemeth specification into Word so that a more accessible pdf could be created in English. A partial version with MathSpeak functionality is given in The Nemeth Braille Code for Mathematics and Science. An ASCII braille version (brf) can be downloaded from here.

Unicode 9.0 has a total of 2310 characters that have the math property (see Math property in DerivedCoreProperties.txt). Of these, many of the more advanced symbols don’t have unambiguous Nemeth representations. In particular, Nemeth doesn’t distinguish between slanted and vertical bar overlays, e.g., 219A ↚ and 21F7 ⇷ are both given by ⠳⠈⠫⠪⠒⠒⠻ . Nemeth doesn’t have white arrows like ⇦, and white/black arrow heads like ⭠. Unicode has symbols like the bowtie ⧑ that don’t have apparent Nemeth representations. One possibility for ⋈ is as a shape with a suggestive name like “bt” as in ⠫⠃⠞, but one still needs to encode whether the sides are black or white since Unicode encodes all four possibilities.

Conversely, Nemeth has quite a few symbols not in Unicode. Often these can be constructed with a combination of Unicode symbols or with math layout objects in applications like Microsoft Word. We can submit proposals to add characters given in the Nemeth specification but not yet in Unicode provided the characters occur in journals or books. Nemeth’s extended tilde is one such character to research. Since Nemeth braille is based on a productive syntax, many symbol combinations can be created. Eventually I hope to collect all Unicode math characters that have reasonable Nemeth braille sequences and add their mapping data to the information associated with Unicode Technical Report #25, Unicode Support for Mathematics.

The table below lists some representative Unicode characters used in mathematical text along with their Unicode names and the corresponding Nemeth math braille sequences. The table doesn’t include any mappings of the Unicode math alphanumerics, since they are defined in the post Nemeth Braille Alphanumerics and Unicode Math Alphanumerics. Relational operators (Nemeth calls them “signs and symbols of comparison”) need to be surrounded by spaces. The spaces are not included in the table since the relational operator property is defined in the MathClass data file and software can insert spaces programmatically. It’d be easy to add the math behavior column in MathClass.txt for quick reference. The full mapping table isn’t included since I don’t know how to convince the MSDN blogging facility to use a braille font that displays nondot place holders and braille is hard to read without them.

]]>Furthermore, in text processing, it’s important to think of the IP as in between characters, since a character you type could end up in the preceding text run, in the following text run or in a text run of its own. Consider an IP that immediately follows the word bold in “**bold**text”. Will the next character you type be bold or not? The answer depends on whether the IP has the formatting of the preceding text run, the formatting of the following text run or some other formatting chosen in part using toolbar format commands or hot keys like Ctrl+B.

Quantitatively, we describe the insertion point by its character position (cp). In the following figure, character positions are represented by the lines separating the letters. The corresponding cp values are given beneath the lines. The text run starting at cp 5 and ending at cp 7 contains the two-letter word “is”. The difference between these cp’s is 2, the count of characters in the text run.

The cp 5 also marks the end of the text run starting at cp 0. We can call such a cp an “ambiguous cp”, since it delimits two adjacent text runs. This ambiguity is illustrated for an IP in between “**bold**” and “text” above.

In BiDi text, such as a mixture of Arabic and English text, the directional ambiguity of an IP between text runs of opposite directionality is usually resolved by the directionality of the current keyboard language. This choice is made because the purpose of the IP is to reveal where the next character typed will be entered. This works unambiguously in BiDi text except when the IP follows a digit and the keyboard is right-to-left. This is because consecutive digits are invariably displayed left-to-right, rather than right-to-left (except for N’Ko). If you then type a digit with a right-to-left keyboard, the IP will be displayed to the right of the digit, but if you instead type a right-to-left letter, that letter will be displayed to the left of the digit(s). There’s no way to know what the user will type next, so in this scenario the IP may not be displayed where the next character typed will be inserted. The bottom line is that in BiDi text you can’t intuit desired behavior 100% of the time, partly because people have conflicting needs. The choices were made because they work the clear majority of the time.

If the top of the caret (text cursor) has a tick mark that points left, an RTL (right-to-left) keyboard is active, while a tick mark that points right indicates an active LTR keyboard. Office apps use a tick-mark-free caret for LTR keyboards. If a document doesn’t have any BiDi text, such a caret is unambiguous. If a document does have BiDi content, some users might prefer to see the right-pointing tick mark on the caret when the keyboard is LTR. Office used to have such an LTR caret, but abandoned it back in the last century. Having the RTL caret was considered sufficiently different from a tickless caret to resolve the ambiguities and then there’s less of a hiccup going between pure LTR docs and BiDi documents. I don’t think we’ve gotten negative feedback on this choice. Conceivably, we should have a user option that’s enabled by default in BiDi locales and disabled by default elsewhere. Also, we might want an option to display a shadow caret where a character would be displayed if typed with a keyboard of the opposite directionality.

In the text object model TOM, insertion points consist of degenerate ITextRange or ITextRange2 objects. The insertion point controlled directly by a user editing text is the ITextSelection[2], which is an ITextRange[2] with additional user-interface functionality. If these ranges select one or more characters, they are called nondegenerate. ITextRange methods refer to the cp at the start of a range as Start and to the cp at the end of the range as End. They are retrieved and set by methods like ITextRange::GetStart() and ITextRange::SetStart(). If ITextSelection2::GetCch() returns 0, the user selection is an insertion point. Alternatively the condition End = Start implies an insertion point. If you’re using a degenerate ITextSelection2 or ITextRange2, you can change the character format inheritance by changing the Gravity property. That is typically a bit faster since figuring out which character format properties the IP should use when moved may incur considerable calculation.

The ITextSelection object has its own character formatting properties to allow the user to change the formatting of the insertion point from the formatting deduced from the backing store and selection activity. In RichEdit, all degenerate ITextRange objects have their own character formatting to give clients such flexibility.

RichEdit uses an n/m algorithm to move the insertion point through Latin and Arabic ligatures and to select part way through such ligatures. The rationale for doing this is given in Ligatures, Clusters, Combining Marks and Variation Sequences. It would confuse most users if → moved past a whole ffi ligature in the word “difficult”. So, caret motion cannot be dictated solely by the font.

People need to know where the IP is so that they know where they enter text. This applies to all editing mechanisms including speech and braille displays. As described in Speaking of math…, the insertion point is identified by speaking the character or object at the insertion point. Most braille displays have 8 buttons for inputting 8-dot braille. However, the popular braille systems only use 6 dots. That leaves dots 7 and 8 available for innovative purposes, such as marking the IP.

VoiceOver indicates the position of the text cursor (IP) on braille displays by flashing dot 8 of the braille cell preceding the IP and dot 7 of the braille cell following the IP. This locates the IP in between two braille cells, much as the text cursor on screen displays the IP in between two characters. Voice over also raises dots 7 and 8 to show the position of the VoiceOver cursor, to help you find it within the line of braille.

Finally, I can’t resist mentioning a cool editing feature of the Visual Studio editor, namely the multiline IP. Create this with a mouse by Alt+dragging the IP over multiple lines. I use it all the time to delete and insert text in whole columns. You can also use alt+dragging to select and operate on blocks of text, e.g., to delete or copy them.

]]>A new version of Unicode Technical Note #28, *UnicodeMath, a Nearly Plain-Text Encoding of Mathematics* is now available. It updates several topics and references and uses the name UnicodeMath instead of Unicode linear format. Since there are several math linear formats, such as Nemeth braille, [La]TeX, and AsciiMath, having the name UnicodeMath clarifies the discussion nicely. The text has been polished in other ways too and some errors have been corrected. No notational constructs have been added, so the version number is only incremented to 3.1.

Here’s a UnicodeMath example in case you don’t want to read the whole spec ☺ The formula

sin θ=(e^iθ-e^-iθ)/2i

displays as

Operators and operator precedence are used to delimit arguments. A binary minus has lower precedence than the superscript operator ^ and the fraction operator /, but a unary minus has higher precedence than ^. This approach contrasts with LaTeX and AsciiMath which require that arguments consisting of more than one element be enclosed in {} or (), respectively. In LaTeX, the formula above is given by

\sin\theta=\frac{e^{i\theta}-e^{-i\theta}}{2i}

In AsciiMath, the formula is given by

sin theta=(e^(i theta)-e^(-i theta))/(2i)

In Microsoft Office apps, you can enter Unicode symbols in UnicodeMath using the corresponding [La]TeX controls words such as \theta, using names that you choose, or using symbol galleries.

]]>Let’s start with the Nemeth braille approach used by MathSpeak, though it doesn’t use braille codes. Nemeth braille uses subscript/superscript level shifters. For example,

A sub/sup level stays active until another level is met. The level shifter to go back to the baseline is “baseline”. This is the way the blind mathematician Abraham Nemeth liked to have people speak subscripts and superscripts to him. Back in his day, computer math speech wasn’t available and people read math to him. His sub/sup speech is efficient and unambiguous. He didn’t even say *x*^{2}_{ }as “x squared”, but as “x sup 2”.

This has the advantage that if *x*^{2} is the second component of the vector **x**, it isn’t misidentified as “x squared”. Superscripts don’t always mean powers. For example, the triple scalar product **a****⋅**(**b×c**) of the vectors **a**,** b**,** **and** c** is given by *ε _{ijk} a^{i} b^{j} c^{k}*, where

Furthermore, the level of nested subscripts/superscripts is always clear with level shifters. On the other hand, saying “e to the minus x squared” gives the meaning of that expression without any parsing. A more verbose version of the Nemeth approach is to say “superscript” and “subscript” instead of “sup” and “sub”. Saying the complete words is helpful at first. But as you get familiar with it, the three-letter abbreviations are faster and easier to follow. Too much verbiage gets in the way of comprehending math.

Except for Nemeth himself, the references linked to at the start of this post all say *x*^{2} as “x squared” and *x*^{3} as “x cubed”. Superscripts as indices aren’t common and a little AI could recognize them. ClearSpeak says *x ^{n}* as “x to the nth power”, while I prefer “x to the n”. “nth” requires localization, whereas ‘n’ alone does not. In my lectures on physics over the years, I don’t think I ever added the word “power”. Although grammatically correct, it wastes time, and being grammatically correct isn’t necessarily a goal for math speech. Math speech wants to be efficient and unambiguous, but some degree of abbreviation helps convey the semantics more efficiently. In fact, mathematics owes a significant part of its success to its concise notations. A side benefit of using abbreviated speech is that localization is simplified: you don’t have to worry much about word order differences and declensions.

If you don’t use the sub/sup/base level shifters, how do you handle compound subscripts and superscripts unambiguously? The various math linear formats except for Nemeth braille all handle compound scripts using tree structures such as TeX’s a^{b_2} or UnicodeMath’s a^(b_2) for *a ^{b₂}*. One could speak these characters, but it’s better to speak what they represent since {} and () are used for a variety of syntactic purposes and may be nested. Accordingly, one can say “a to the b sub 2 end sup”. Here UnicodeMath’s ‘(‘ is replaced by “to the” and the ‘)’ is replaced by “end sup”. For

Numeric fractions like ¼ are spoken as “one fourth” and simple fractions like a/b are spoken as “a over b”. A fraction is compound if it contains one or more operators with lower precedence than division, such as (a+b)/c. For compound fractions, the beginning and end of the fraction need to be spoken to differentiate between expressions like a/(b+c) and a/b + c. If you say “a over b plus c” it means a/b + c, since we adopt the usual convention that division has higher precedence than addition. It also helps to pause a bit before saying “plus c”.

In the spirit of announcing the start and end of compound entities, one might want to speak a compound numerator as “numerator…end numerator” and a compound denominator as “denominator…end denominator”. But both ClearSpeak and MathSpeak prefer to speak a compound fraction as

“start fraction <numerator> over <denominator> end fraction”.

This is similar to TeX’s notation “{<numerator>\over<denominator>}” and to the Nemeth braille fraction ⠹ <numerator> ⠌ <denominator> ⠼ . This choice is more efficient when both numerator and denominator are compound. Both approaches allow nesting of fractions. Briefer choices include “frac…over…end frac” and “b frac…over…e frac”.

The last of these is how Abraham Nemeth liked fractions to be spoken. Furthermore, if a fraction contains another fraction, he’d say “b b frac … o over … e e frac” for the outer fraction and “b frac…over…e frac” for the inner fraction. He’d repeat the ‘b’, ‘o’, and ‘e’ as many times as the deepest fraction’s nesting level, like stuttering. MathSpeak has a similar option that uses “start” for ‘b’, “over” for ‘o’ and “end” for ‘e’. Revealing the nesting levels is similar to the way we speak nested parentheses as “open paren”, “open second paren”, “open third paren”, and so forth as in ClearSpeak, but in opposite nesting order. MathSpeak and Nemeth Braille indicate the nesting level of square roots and other roots, but don’t give a way to indicate the nesting level of parentheses.

One ends up with a plethora of choices. Since different folks like different choices, both MathSpeak and ClearSpeak offer several speech options. Some choices can be handled by a verbosity level. But qualitatively different choices might best be handled with settings in a dialog box. Nemeth sub/sup level shifters versus tree speech of compound scripts is an example of the latter. See also Larry’s Speakeasy, which gives English speech for a wide variety of mathematics.

]]>

In addition to being the most readable linear format, UnicodeMath is the most concise. It represents the simple fraction, one half, by the 3 characters “1/2”, whereas typical MathML takes 62 characters (consisting of the <mml:mfrac> entity). This conciseness makes UnicodeMath an attractive format for storing mathematical expressions and equations, as well as for ease of keyboard entry. Another comparison is in the math structures for the Equation Tools tab in the Office ribbon. In Word, the structures are defined in OMML (Office MathML) and built up by Word, while for the other apps, the structures are defined in UnicodeMath and built up by RichEdit. The latter are much faster and the equation data much smaller. A dramatic example is the stacked fraction template (empty numerator over empty denominator). In UnicodeMath, this is given by the single character ‘/’. In OMML, it’s 109 characters! LaTeX is considerably shorter at 9 characters “\frac{}{}”, but is still 9 times longer than UnicodeMath. AsciiMath represents fractions the same way as UnicodeMath, so simple cases are identical. If Greek letters or other characters that require names in AsciiMath are used, UnicodeMath is shorter and more readable.

Another advantage of UnicodeMath over MathML and OMML is that UnicodeMath can be stored anywhere Unicode text is stored. When adding math capabilities to a program, XML formats require redefining the program’s file format and potentially destabilizing backward compatibility, while UnicodeMath does not. If a program is aware of UnicodeMath math zones (see Section 3.20 of UnicodeMath), it can recover the built-up mathematics by passing those zones through the RichEdit UnicodeMath MathBuildUp function. In fact, you can roundtrip RichEdit documents containing math zones through the plain-text editor Notepad: the math zones are preserved!

As its name implies, AsciiMath uses only ASCII characters, although it converts to MathML with access to a much larger character set. AsciiMath is relatively simple to parse and can handle many mathematical constructs. AsciiMath shares some methodology with UnicodeMath, such as eliminating the outer parentheses in fractions like (a+b)/c when converting to built-up format. AsciiMath is designed to work with a MathML renderer, such as MathJax. In Microsoft Office apps, UnicodeMath builds up to the LineServices math internal format, which represented externally by OMML.

By default, the Office math autocorrect facility contains most [La]TeX math symbol control word definitions such as \beta for β. AsciiMath has a subset of such control words but omits the leading backslash. The user can modify such control words in the Office math autocorrect list or add them explicitly, but it’d probably be worth adding an option to make the leading backslash optional. That would speed up keyboard entry of UnicodeMath via math autocorrect. The RichEdit dll includes the UnicodeMath build up/down facility as well as converters for other math formats, such as MathML and OMML. It would be straightforward to add an option to the RichEdit UnicodeMath facility to accept AsciiMath input in general. Such an option would be handy for people that know AsciiMath.

One C++ oriented autocorrect choice in AsciiMath is that typing != enters ≠. Although I program in C++ almost every day, I think /= is a better choice for entering ≠. For one thing, using != for ≠ complicates typing in an equation like n! = n(n-1)(n-2)…1, which is the main reason we didn’t implement it. But in Office apps this equation can also be entered by typing ! = instead of !=, since math spacing rules insert space between ! and = and the RichEdit UnicodeMath facility automatically deletes a user’s space if typed there (see User Spaces in Math Zones). So, that’s an easy work around for entering an n! equation if one wants to support != for ≠. The RichEdit UnicodeMath facility supports most Unicode negated operators by sequences of / followed by the corresponding unnegated operator as described in the post Negated Operators.

<gripe> Meanwhile the C++ language should recognize ≠, ≤, ≥, and ≡ as aliases for !=, <=, >=, and ==. It seems primitive that C++ doesn’t do so in this Unicode age of computing. At least the C++ editing/debugging environments should have an option to display !=, <=, >=, and == as ≠, ≤, ≥, and ≡. </gripe>

Here’s a table with various formats for the integral

Format |
Representation |

UnicodeMath | 1/2𝜋 ∫_0^2𝜋▒ⅆ𝜃/(𝑎+𝑏 sin𝜃 )=1/√(𝑎^2−𝑏^2 ) |

AsciiMath | 1/(2pi) int_0^(2pi) dx/(a+bsin theta)=1/sqrt(a^2-b^2) |

LaTeX | \frac{1}{2\pi}\int_{0}^{2\pi}\frac{d\theta}{a+b\sin {\theta}}=\frac{1}{\sqrt{a^2-b^2}} |

Note that UnicodeMath binds the integrand to the integral, whereas AsciiMath and LaTeX don’t define the limits of the integrand. The Presentation MathML and OMML for this integral are too long to put into this post.

There is a unicode-math conversion package for Unicode enabled XeTeX and LuaLaTeX. The name UnicodeMath seems sufficiently different from unicode-math that there shouldn’t be any confusion between the two. The unicode-math package supports a variety of math fonts including Cambria Math, Minion Math, Latin Modern Math, TeX Gyre Pagella Math, Asana-Math, Neo-Euler, STIX, and XTIS Math. Did you know there are so many math fonts?

Enjoy the new name UnicodeMath. I am and it already appears near the end of my previous blog post, Nemeth Braille Alphanumerics and Unicode Math Alphanumerics. If you’re interested in the origin of UnicodeMath, read the post How I got into technical WP. The forerunner of UnicodeMath originated back in the early microcomputer days and had only 512 characters consisting of upright ASCII, italics, script, Greek and various mathematical symbols used in theoretical physics. Unicode 1.0 didn’t arrive until 10 years later.

]]>