Occasionally people ask how foreground (text) and background colors work in RichEdit. This post gives an overview of RichEdit colors in a variety of scenarios including default coloring, formatted coloring, selection coloring (both normal and acetate), math-zone highlighting and special draft and high-contrast modes. It also describes refinements such as switching to XOR coloring when the foreground and background colors are too similar. Interestingly enough except for math-zone highlighting, all these coloring options work with RichEdit plain-text controls as well as with rich-text controls. The colors are character formatting properties and some of the discussion applies to all character formatting properties. Background colors for table cells are described in RichEdit's Nested Table facility.
Default colors (autocolor)
Unless overruled by explicit formatting, RichEdit gets its text and background colors by calling ITextHost::TxGetSysColor(iColor), where iColor is a color index like COLOR_WINDOW. Typically the RichEdit host delegates this call to the Windows GetSysColor() function, but the callback gives the RichEdit host the opportunity to return other colors instead. For example, by default the background color is given by calling ITextHost::TxGetSysColor(COLOR_WINDOW). The default text (foreground) color index is COLOR_WINDOWTEXT. These and some other color types are defined in winuser.h. In a windowless RichEdit control the host can choose all such colors.
In a window’d RichEdit control, the built-in host’s ITextHost::TxGetSysColor(iColor) generally calls the operating system’s GetSysColor(iColor). There are two exceptions. 1) If an EM_SETBKGNDCOLOR message is received, the color supplied by that message is used for COLOR_WINDOW instead of calling GetSysColor(). 2) If the window style is disabled (WS_DISABLED), by default the text color is given by GetSysColor(COLOR_GRAYTEXT) and the background color is given by GetSysColor(COLOR_3DFACE). But if an EM_SETDISABLECOLORS message is received, the text color is given by the message’s wparam and the background color is given by the lparam.
When a RichEdit control is instantiated, it creates a default character format structure CHARFORMAT2 with the foreground color effect CFE_AUTOCOLOR and the background color effect CFE_AUTOBACKCOLOR. Whenever these autocolor effect settings are active, RichEdit calls ITextHost::TxGetSysColor() for the default text and background colors, respectively. After initializing the CHARFORMAT2 structure, RichEdit calls ITextHost::TxGetCharFormat() to allow the host to overrule the default structure properties. For example, the host can reset the CFE_AUTOCOLOR and/or CFE_AUTOBACKCOLOR effects, which changes the default colors to be the default CHARFORMAT2::crTextColor and/or CHARFORMAT2::crBackColor colors, respectively, instead of those given by calling ITextHost::TxGetSysColor(). After initialization, the default character format can be changed by sending an EM_SETCHARFORMAT message (use ITextServices::TxSendMessage for windowless controls) with wparam = SCF_DEFAULT (but not SCF_SELECTION) and lparam = CHARFORMAT2 *. It can also be changed by calling ITextDocument2::SetDocumentFont().
Text run colors
Similarly, an EM_SETCHARFORMAT message can set the text and/or background colors on the currently selected text run. For this, wparam = SCF_SELECTION and lparam = CHARFORMAT2 *. The text-run color can also be set by calling ITextFont::SetForeColor(), where the ITextFont object is obtained by calling ITextRange::GetFont() and the ITextRange selects the text run. Here any ITextRange can be used, which is handy since the call doesn’t change the user selection unless the ITextSelection range itself is used. The background color of a text run can be set on a run by calling ITextFont::SetBackColor().
The text and background colors for selected text depend on whether acetate selection (described next) is active. If acetate selection isn’t active, the selection text color is obtained by calling ITextHost::TxGetSysColor(COLOR_HIGHLIGHTTEXT) and the selection background color is obtained by calling ITextHost::TxGetSysColor(COLOR_HIGHLIGHT).
With acetate selection, the text color remains the same whether or not the text is selected except when the resulting text color is too close to the acetate background color to be read easily. The acetate background color is a blend of the background color and the color returned by ITextHost::TxGetSysColor(COLOR_HIGHLIGHT).
More precisely, colors are defined by red, green and blue components in the COLORREF RGB DWORD defined by
The acetate background red color is given by a fraction (f) of the background red (r) component plus the complementary fraction (1 − f) of the COLOR_HIGHLIGHT red component. Ditto for the green and blue components. Typically 39% of the COLOR_HIGHLIGHT RGB components are mixed with the corresponding background color components to get the acetate background color.
Office applications use acetate selection, whereas Windows often doesn’t. One advantage of acetate selection for RichEdit is that partial ligature selection doesn’t get clipped by the rectangular selection background. This is illustrated in the following image where the f of an fi ligature is selected
Since the f’s text color doesn’t change when selected, the f’s underhang and overhang aren’t clipped. In contrast without acetate selection, RichEdit appears to select the whole fi ligature and clips the f’s underhang since RichEdit doesn’t have the code to render the ligature glyph three times with appropriate text colors. You can try out the selection with an Arabic ligature too, e.g., type a lam aleph (لا – gh with an Arabic keyboard) and then shift+→ to select the lam alone. You’ll see the acetate highlight go half way thru the لا. Acetate selection works the same way as in Word.
Acetate selection is used by default in RichEdit. To disable acetate selection, send EM_SETEDITSTYLEEX with wparam = lparam = SES_EX_NOACETATESELECTION.
Underlines are displayed in the text color by default. But ITextFont::SetUnderline(value) can choose one of 16 COLORREFs as well as specifying an underline type. The MSDN documentation doesn’t yet say how the colors are defined, although the capability has been working for many years. Specifically the low byte of value specifies the underline type as documented in MSDN and the next byte has a color index. If that byte equals 0, then the text color is used for underlines. If it’s 1 up to 16, it’s a color index that accesses one of 16 COLORREFs defined in ITextDocument2::GetEffectColor(). The companion function ITextDocument2::SetEffectColor() can be used to change those color values if desired. As an example, ITextFont::SetUnderline(0x600 + tomDash) defines a red, dashed underline.
RichEdit has a special draft mode enabled by sending EM_SETEDITSTYLE with wparam = lparam = SES_DRAFTMODE. When enabled, all text is displayed with the font and font height given by ncm.lfMessageFont.lfFaceName and ncm. lfMessageFont.lfHeight, where ncm is obtained by
ncm.cbSize = sizeof(NONCLIENTMETRICS);
SystemParametersInfo(SPI_GETNONCLIENTMETRICS, sizeof(NONCLIENTMETRICS), &ncm, 0);
The draft-mode text color is obtained by ITextHost::TxGetSysColor(COLOR_WINDOWTEXT).
An empty RichEdit control without the UI focus can display place-holder text such as “Type answer here”. Such text is entered into the control by the EM_SETTEXTEX message with the ST_PLACEHOLDERTEXT flag (16) and it is displayed with the color for COLOR_GRAYTEXT. As soon as the control gets the focus, the placeholder text disappears.
IMEs and proofing tools like to use character formatting such as squiggly underlines and special colors to indicate specialized text treatment, e.g., a spelling error. To avoid changing the underlying character formatting, RichEdit provides temporary formatting of underline type and color, and foreground and background colors. When such formatting is removed, the original formatting is used again for display. Temporary formatting isn’t persisted in file formats and can’t be read out by the client, only applied.
To define temporary formatting properties, call ITextFont::Reset(tomApplyTmp) or ITextFont::Reset(tomApplyIMETmp for proofing or IME purposes, respectively. Then specify the temporary formatting colors by calling ITextFont::SetForeColor() and ITextFont::SetBackColor() as desired. Specify the temporary underline color and type by calling ITextFont::SetUnderline(value). Specifically if value = tomAutoColor, autocolor (default text color) is used. If (0xFF000000 & value) = 0xFF000000, the temporary underline color is set to 0x00FFFFFF & value. Else if value is a valid underline type, the temporary underline type is set to value. To apply the temporary formatting so defined, call ITextFont->Reset(tomApplyNow). If a temporary formatting property isn’t defined, the corresponding original property is used.
High contrast and colored fonts
RichEdit doesn’t have a special high contrast mode. So long as default colors are used, the RichEdit host can return high contrast colors for text, background and selection coloring. Furthermore in D2D/DWrite mode, the RichEdit host supplies an ID2D1RenderTarget (see ITextServices2::TxDrawD2D) that the host itself usually implements, calling D2D functions as necessary. If so the host can change the colors as desired on the fly. In particular, the host can force all text colors to be a specific high-contrast value and it can support colored fonts, such as Segoe UI Emoji.
If the text and background colors are too similar, the text color is changed to be visible relative to the background. The criterion for being too similar is that the squared length between the two colors in the three-dimensional RGB space is less than 4% of the maximum squared length. There may well be better criteria, but this seems to work pretty well in practice. If the colors are too similar, other possible text colors may be tried, but the ultimate choice is the XOR of the background color, i.e., use a text color given by the inverse of the background color (XOR each RGB bit with 1). This converts black into white, red into cyan, blue into yellow, etc., giving considerable contrast for most people.
Math-zone highlighting and coloring
When the selection is an insertion point (IP) in a math zone that has a default background color, relevant sections of the math zone are highlighted with a gray background. If the IP is not inside a math object, the whole math zone is highlighted with the light gray background RGB(232, 236, 239). If the IP is inside a math object, the object alone is highlighted with the light gray background and the argument of the object is highlighted with the slightly darker gray background RGB(215, 220, 230). That way the user can see which math object and argument are involved. For example, if the IP is in the numerator of a fraction, the numerator is highlighted with the darker gray background and the rest of the fraction is highlighted with the lighter gray background as in
This differs from Word, which only displays the argument with a gray background. Word encloses the whole math zone in a border, which makes double highlighting less necessary. PowerPoint and OneNote use RichEdit to display math zones, so they have double highlighting. Selection inside a math zone uses the same coloring as outside a math zone. Math accents, fraction bars, enclosure borders, integral signs, etc. are displayed with the color assigned to the math object start delimiter.
RichEdit has automatic hyperlinks and friendly-name hyperlinks. Automatic hyperlink text is normally displayed in blue with a blue underline. But if blue is too similar to the background color, the color for COLOR_WINDOWTEXT is used. By default, friendly-name hyperlink text is also displayed in blue with a blue underline unless the name text is formatted with an explicit color. Explicit formatting takes precedence. The client can suppress the automatic blue coloring for friendly-name links by sending EM_SETEDITSTYLE with wparam = 0 and lparam = SES_EX_HANDLEFRIENDLYURL.