New Experimental Console Features


As of Windows 10 build 18298, when you open the properties page of any Console window, you’ll notice an additional “Terminal” tab. Click on this tab and you’ll see several new settings for some experimental features we’re working on for future OS releases:

Experimental settings in action

Experimental settings in action

Important Note: These settings are "experimental", because in certain scenarios, it's possible that they might not behave like you would expect them to, may not make it into the next OS release, and may change completely before we ship! But have fun with them anyway and let us know how you get on – we’d love to hear your feedback.

Let’s outline each of the settings groups from the bottom up, ending with a more thorough discussion the Terminal Color settings:

Terminal Scrolling

Disable Scroll Forward:

  • When cleared (default state), the Console can to scroll anywhere within the Console’s buffer, even below the most recent row of text displayed
    When checked, the Console will not allow scrolling below the last line output, similar to how most *NIX terminals work

Cursor Shape

Legacy Style: Thicker underscore by default, but controlled by the cursor size property in the "Options" page

Underscore: Thinner underscore

Vertical Bar: For all you crazy kids who like your cursors tall and skinny

Empty Box: The second-best cursor:

Solid Box: The one true cursor

Cursor Colors

Inverse Color: The current and default behavior where the cursor color is the calculated inverse of the background color … which results in, for example, this adorable fetching pink against the bright green background of my first computer – the Dragon 32 😊

 

Use Color: Draw the cursor in a specific RGB color, like this fetching green against dark gray background:

 

Terminal Colors

The Terminal Colors fields allow you to specify specific RGB color values for the default foreground and/or background colors. These colors are independent of the color palette and are not affected by changes you make to your color palette! But before we discuss the inherent implications here, we need to consider how these new settings fit into the color changes we’ve been making over the last several Windows 10 releases:

Karma Karma Karma Karma Console Chameleon

Over the last couple of years, during “The Grand Overhaul” of Windows Console, we’ve made some significant improvements to Console’s parsing and handling of ANSI/VT sequences, and how the Console stores and renders colored text.

This all started with a rewrite of the Console’s VT parser. Prior to the (then new) team’s efforts, Windows Console barely supported any VT sequences at all. But with the steady growth in adoption of *NIX-first open-source tools, many of which emit VT sequences to color and draw text-based UI’s, it was increasingly imperative that the Windows Console be able to support VT sequences.

Before (XP) and after (Win10) VT support added to Console

Before (XP) and after (Win10) VT support added to Console

This was made even more important and urgent due to the arrival of Windows Subsystem for Linux which allowed unmodified Linux binaries to run natively on Windows, allowing Windows users to easily run most of the *NIX tools they’d previously had to fire-up in a separate VM.

The first significant version of the Console’s new VT engine shipped in Windows 10 1609 (Anniversary Update), and was quickly followed in the 1703 (Creators Update) release which contained much richer support for a wider range of VT sequences. This allowed Console to render many of even the most complex text-based UI’s generated by tools like Tmux (a great tutorial here for the uninitiated) and Midnight Commander … or even vim, bash, Midnight Commander and htop running simultaneously in separate Tmux panes:

TMux hosting vim, bash, Midnight Commander, and htop

TMux hosting vim, bash, Midnight Commander, and htop

Also in Creators Update, the Console shipped its massively improved color support, increasing its color support from a mere 8-bit/16-colors (a single 8-bit value held both the foreground and background color 0-7 in the low and high nybble respectively), to full 24-bit/16M-colors!

But we weren’t yet done! In the Fall Creators Update (Oct 2017), we updated the Console’s default color palette for the first time … ever! After working with Microsoft’s usability and accessibility teams, and following recommended color selection guidelines, we arrived at a new color palette that is FAR better suited to modern high-contrast displays.

In addition, we also shipped an open-source ColorTool which you can use to apply different color palette’s to your Console windows if you wish.

So, we’re done with color support in the Console, right?

Nope!

Looking Back, Moving Forward

The Console now lives in two worlds:

  1. 1. Windows – Console supports traditional/legacy command-line Windows apps that call the Win32 Console APIs to color/manipulate text displayed on the Console
  2. ANSI/VT – Console also now supports apps that color/manipulate the text that they display by embedding VT sequences within the text that they emit

Because of #2 above, Console has to support some of the different ways in which VT compatible terminals think about the world.

A key difference between Windows Console and *NIX-based VT Terminals is that *NIX Terminals have a notion of default foreground and background colors, which are independent from the Terminal’s color palette. The Windows Console does not … at least not quite the same!

Windows Console refers the “Screen Text” and “Screen Background” colors to the colors defined in the Console’s palette:

Console color settings

Console color settings

 

But Console has no notion of “default” foreground or background color to the command-line apps that draw on the screen. At best, there’s an assumption that color #0 is often the background color, and that #7 is the foreground, but this is an often-incorrect assumption, not a widely-understood rule. In fact, Windows command-line apps have no way of determining the default foreground and background colors!!

For a good example of this, let’s explore Windows PowerShell: The Windows PowerShell team wanted to differentiate PowerShell Consoles from Cmd Consoles by changing the PowerShell Console’s default background color.

They specifically modified the PowerShell shortcuts’ settings:

  • Re-defined color #5 as a specific shade of blue: RGB(1,36,86)
  • Re-defined color #6 as a specific shade of gray: RGB(238,237,240)

However, they also decided to modify the shortcut’s background and foreground color indexes to

  • Color #5 as the background color (instead of the generally assumed color #0)
  • Color #6 as the foreground color (instead of the generally assumed color #7)

Sometimes, PowerShell wants to, for example, “render this text in <color> on whatever the default background color is”.

Alas, because Windows Console has no API to discover what the “default background color” is, PowerShell assumes that the background is the color defined by color #0 … which usually IS the background color … except in PowerShell which uses color #5

Thus, if you’ve ever mis-typed a command in Windows PowerShell, you’ve likely seen something like this:

PowerShell assumes the background color is #0

PowerShell assumes the background color is #0

Moving Forward

Alas, the changes we’re working on (and exposing via the experimental Terminal settings page) won’t fix this specific issue for Windows PowerShell ☹

Existing/legacy applications that specifically say “draw this text in <some-foreground-color> on color #0” will still draw text in some-foreground-color, on a (usually black/dark) background, even if the Console’s default background color is bright green! In some ways this is good;there will be no behavioral change for existing/legacy apps. But the downside is that such apps will continue to render less than optimal output (like in the PowerShell example above.

However, making Console aware of the notion of “default independent foreground and background colors” will enable Console to more accurately render VT enriched text-UI, and will give Console users more control over the customizability of their Command-Line environment.

Call To Action!

This issue further amplifies the recommendation to start writing (or update existing) Command-Line apps to emit VT enriched text, rather than calling Win32 Console APIs to control/format Console output: VT-enabled apps will not only benefit from rendering improvements like those described above (with more on the way), but they’ll also enjoy richer features available via VT-sequences, like 24-bit color support, and default foreground/background color support.

 

We look forward to hearing how you get-on with these experimental changes. If you find issues with the color support, please find/file issues on our GitHub issues repo, reach out and let @richturn_ms know on Twitter, and/or submit feedback via the Feedback Hub.

 

Comments (51)

  1. JJZakius says:

    I hope you’ll continue to work with ConEmu (and/or cmder, as ConEmu seems to be stale a bit) to provide the best console experience. For now it doesn’t work really good with some of the newest features but it still is the best combo anyone can wish for.
    Besides that, while I am aware we have WSL for a reason it would be really nice for us if you reached out to devs of these few tools that don’t work natively yet and work together to bring them to Windows without the need of WSL.
    The last issue, possibly most complex one, comes from the fact, that Windows has some CLI tools that behave differently than *NIX ones with the same name (personally I stumbled upon find) making some perfectly fine scripts/tools not work properly.
    Unfortunately I have no one size fits all solution for this (as many legacy scripts and tools aimed for Windows may rely on Windows argument syntax and even trying to guess which one was used may cause unpredictable consequences)

    1. We do actually work with the developers of ConEmu and several other widely re-used libraries and tools, helping them adopt some of our newer features. It takes a little time between a feature being introduced and it being adopted and released, but it’s underway.

      And yes, you pretty much explained why we cannot arbitrarily change the names of tools, or the types/names/order of arguments they use – if we do, we tend to break A LOT of scripts, tools, processes, etc. For example, the `find` in Windows today is compatible with the find command added to MS-DOS 2.0 back in 1982!

    1. News in what way? It shipped in 1809.

  2. mnmicrosoft says:

    I have been using the new 24-bit color VT sequences for over a year in a quite sophisticated way (with the side effect of quite complicated code to support it, as a fully addressable, windowed, 2D text surface). But having a background in graphics, it seems there should also be full support for “plotting” characters and colors a la pixels. I seem to recall the Win32 console API supporting this “2D” notion of the console quite naturally and directly. I don’t see why we shouldn’t have both. Can’t the console API be extended or enhanced to support the new capabilities? Indeed it seems silly to have to indirect through VT codes when both the app and the underlying OS think of the display as a 2D grid of cells. I know the VT codes are fully capable of emulating this, but it is a verbose and low-performance way of accomplishing what is conceptually simply storing a value in an array.

    1. Adding / extending an API has a major long-term support and maintenance cost that, frankly, isn’t worth it considering that pretty much every other platform’s command-line infrastructure is based around embedding ANSI/VT sequences into text streams.

      While this may seem a little obtuse at first, also remember that ANSI/VT enriched text also traverses VPNs, SSH connections, most other forms of communication protocol, etc., whereas API calls do not. Thus, if your app emits ANSI/VT enriched text, any Terminal on any platform should be able to render it correctly when remoting a Windows host machine.

      1. mnmicrosoft says:

        I understand your point but this seems an unusual and specific perspective. Usually there is a trade off between native APIs and cross-platform techniques — native is superior in some way at the expense of portability, else it has no reason for being. DirectX 12 is far superior to HTML5 if you are trying to render something on a Windows machine and want or need that performance or flexibility. If as a developer I am targeting a platform, with no interest or intention of portability, the native API wins by default. I don’t want to even risk losing any performance or functionality now or in the future just for the sake of portability I don’t even use. Here, we have a very unusual reversal where the native API is /inferior/ to the portable one, by choice, when it could most certainly be made equivalent or superior if the will was there. I know my currently in development true-color Windows console app would be simplified if I had that choice. It seems the perspective being adopted is prioritizing for the developer, making assumptions about use cases that are not true for all of us.

  3. SMFX says:

    I’m glad to see them putting in some effort to make more features, but I find it a little frustrating that they put all this effort into expanding support to garner compliance from another platform & the same article point out a shortcoming in the existing native format that could also be expanded. Crazy idea: expand Win32/64 API,. Net, or other library to support these settings too.

    1. We hear you and we toiled over this for quite a while. As I replied to other similar questions herein:

      Adding / extending an API has a major long-term support and maintenance cost that, frankly, isn’t worth it considering that pretty much every other platform’s command-line infrastructure is based around embedding ANSI/VT sequences into text streams.

  4. It would be great if the Win32 Console API were modified to output the VT escape sequences with new versions of the console. It would be really great if it was extended to support all available VT sequences, rather than have everyone write their own VT console library.

    1. It’s actually not quite as simple as that:

      To work correctly, when you call the API to change color, write text, etc., a state engine needs to keep track of cursor position, etc. in a locally hosted buffer, and then periodically render “dirty” regions of the buffer to the connected terminal as a series of text & VT. This is actually what the newly added ConPTY infrastructure actually does!
      There are some ANSI/VT sequences that have no API equivalent

      However while some of these issues could be partly remedied in the short-term, there’s a significant cost in terms of maintenance and support to adding additional new OS APIs for a Windows-only command-line apps vs. building frameworks and tools to make it easier to build such apps at the app-level (e.g. TurboVision).

  5. Fworg64 says:

    Colors are good, are there plans to overhaul tab complete in the near future?

    1. We are looking at providing an improved settings & config mechanism in the future.

      1. Bash completion support for Command Prompt would be absolutely amazing.

        1. Sorry – we’re unable to add completion to Cmd: Cmd is very old and incredibly fragile. Even the smallest change to Cmd ends up breaking someone, somewhere. Instead, we strongly recommend using PowerShell which already has good auto-completion support, but is also actively under development, runs on Windows, Mac, and Linux, and is incredibly … well … PowerFul 😉

  6. In /usr/bin/gnome-terminal there is the possibility to start a new window from an existing window, and also the possibility to have more than one tab in a single window. At least the possibility to start a new window should be present in the new Console Feature. Thanks.

    1. You can open a new Console from a currently open Console by calling one of the following:
      Cmd: `start`
      PowerShell: `start PowerShell`

      We ALMOST got multiple tab support for free with the exploratory Sets & Tabs feature, but it was withdrawn part way through the 1809 product cycle. We are, however, keen to add tab support in the future.

      1. Rich, thanks for your kind and quick answer! It happens to me quite often to start a new window, and I will sure use the method you indicate.

  7. skSdnW says:

    What is stopping you from adding a function that let’s classic Win32 apps determine the fore/back colors? ConsoleQuerySetting or whatever.

    1. The point being that in the world of VT, nobody cares what the default fg/bg colors are – they just emit text wrapped in VT that says “draw this in the default fg/bg colors”.

      Introducing new API’s to allow apps to discover such settings leads to innumerable errors involving apps reading the colors once at startup and then getting them wrong when a user changes color afterwards. And then we’d have to maintain an API that introduces incorrect behavior and assumptions … for decades.

      While it sounds a little draconian without context, the right thing to do is to adopt/migrate-towards the VT way of doing things. We’ll write-up a detailed post on this subject in the new-year.

  8. max630 says:

    It should be noted that the linux terminal programs do not usually handle VT sequences themselves but rather use ncurses library which offers higher level interface similar to the Console API.

    1. Many of the more sophisticated apps do. But you’d be amazed at how many don’t and simply emit formatted VT strings!

  9. SparkX120 says:

    I have been using tmux and other programs that use VT escapes in wsl with native conhost instead of something like mintty for a good while now. It’s really cool to see the progress that has been made on the Windows console. Looking forward to what’s next.

    The one feature I am still really missing though is the ability to hide the scroll bars without nuking the ability to scroll through the buffer with touch or scroll wheel. Every Linux terminal supports doing so and the Mac terminal too.

    1. We agree. Its on our backlog & will get there eventually 😉

      Appreciate your patience 😀

  10. How about the fact that ctrl/Q is broken in 1809. ctrl/q and ctrl/s have been used to stop/start scrolling on VT terminal’s since the 80’s and in the console window too till 1809. SO if you use type to type out text, you stop the output with ctrl/q and start with ctrl/s. sure the pause key works – but what happens when you have a micro minimal keyboard without the pause key ? (like the default touch keyboards)

    1. I am running a post-1809 build, so may be seeing slightly different behavior than you, but I can pause/restart scrolling using CTRL+S (DC1) or CTRL+Q (DC3) respectively when running `ls / -R` on Linux/WSL as expected.

      These sequences don’t work on Cmd/PowerShell because they’d break too many Windows scripts/tools.

  11. Jan Ringoš says:

    Quick question: Is the VT processing enabled per actual console or per output handle to the console? I.e. what happens if two or more programs share the console and one enables VT processing? Assume the other programs are somewhat old.

    1. VT Mode is a property of the Console. If you have two apps simultaneously connected and emitting to the same console instance and one enables VT mode but the other doesn’t then the Console will parse and handle emitted VT.

      In general, everything should still work fine though: The older app is likely to only emit standard text (sans VT) anyhow since was created before VT Mode was added in Win10, and so won’t be affected. And the newer app that does support VT (since it knew to enable VT Mode) should emit correctly formatted VT.

      However, if your VT-emitting app is in the middle of moving the cursor and drawing text on the screen, and the older app writes some text to the Console, it may end up injecting unwanted text into the UX.

      However, I rarely see scenarios like this in Windows environments, though YMMV.

      Read this for more info: https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences

  12. verdy.p says:

    Changes in the console API is still needed to support the VT/ANSI terminal emulator itself in the character-level console I/O.
    Anyway, that emulator should be extended to support also the X11 protocol extension for terminals (like when using Xterm on Linux/Unix). Some extended escape sequences allow working not jsut on character cells but also to define custom characters. Legacy uses were with bitmap glyphs (mapped to some code point or PUAs), but it should be possible to define escape sequences to assign vector-based glyphs or specify glyphs to use from specific fonts and positioning/sizing them in a terminal display cell.
    As well the terminal could have a secondary 2D canvas made visible on selected cells becoming transparent, or layered to allow superpositions of characters or 2D drawable canvas. However I have no idea if a VT-like protocol supports an API-like encoding supporting arbitrary drawing on a canvas (if one was existing, that API should be encoded using SVG, possibly compressed with a common compression algorithm like deflate, without reinventing the wheel) Ideally the console should not jsut be a simple array of character cells but shoul associate one or more background 2D canvasor video layer, and a character grid on top of it. The characters in the cells should also support multicolor glyphs (emojis) and possibly animated glyphs (such as a rotating clock).
    But the character grid should then have to use more than 16-bit encoding: it should now use UTF-32, with one full codepoint in it (and some non UTF-32 pseudo-characters to encode transparent cells with an identifier mapped to a composition of layers, or some extra bits outside the 21 lowest bits, to encode some effects in the 11 available bits in each cell (these bits may be used for accessibility purpose or to mark “protected” cells in input forms, or mark tabulatoion positions, or cells that must not be cleared when clearing the screen, or that must not be scrolled, e.g. for top-bottom bars).
    The console may also need to specify one or several backbuffers (for history, scrolling, or allow movable popups to appear on top without erasing the content of cells overwhich they appear). Special character codes may indicate the backbuffer id and relative position in that backbuffer. All backbuffers would be mapped as a single large array where only one part is displayed in the current scrolling context (each backbuffer would then have its own offset in the global buffer, and its own scrolling context). It would also be useful to have several grids with different cell sizes (notably “double” cells for sinograms, but this could be implemted by allwoing each cell to specifiy sompe additional bits to specify if we display there the left or right part of a “large” character). Composite cells would be useful to allow superpositions and emulation of printed overstrikes. The terminal emulation protocol (VT…) should also be selectable: the transform from the character stream to the console grid would be implemtned in a terminal emulator plugin that would be useful to support multiple emulation protocols and their own escape sequences. and why not having some layers mapped to a full HTML view, or 3D rendering layer? The console support can be much more versatile if it is multilayered and the mulation protocol is also mutable.

    1. Thanks for your thoughts. There’s a lot to unpack here, but this isn’t really a great mechanism to do so, but I’ll summarize some key responses here:

      The Console API remains and supports existing/legacy Windows Command-Line apps. We do not plan to expand the API to duplicate features that are already well supported in a more x-platform friendly manner using VT
      X11 is a GUI remoting protocol and is not something we (the Windows Command-Line team) are looking into, though there may be other teams who’re more interested in this scenario
      VT is a simple encoding purely for describing simple formatting (colors, underline, bold, etc.) to be applied to text in specific cells in a grid-layout – it has no concept of rendering pixels/lines/gradients/transparency/etc.
      All the UI rendering techniques you mention (e.g. protected cells, preventing scrolling etc.) are under the control of the command-line app and is not something that the Terminal can or should provide.
      Modern-day terminals have generally triangulated on supporting XTerm capabilities. Legacy terminal protocols necessary to support legacy hardware/devices are a specialist task best left to specialist software providers, of which there are many who’d be glad of your business 😉
      Storing UTF-32 would massively bloat the memory required for each Console instance, and would murder performance. We know – we tried it!

  13. donmez says:

    These are all nice but can you please fix the text selection to be like in the *nix world? I don’t want to do rectangle selection like a caveman. I just want to select text. 🙂

    1. Do you have “Enable line wrapping selection” checked in your Console’s properties? This was added back in the original release of Windows 10.

      1. Anonymous says:
        (The content was deleted per user request)
      2. donmez says:

        Yes and I do and the copied text is fine. But visually, you are still selecting a rectangle, which feels like Paint to be honest. I love the VT100 improvements and 24bit support but I’d love some text rendering/selection improvements 🙂

  14. What about using DirectX in the background for rendering?
    Is it there?
    I am using FarManager for about 15 years or more.
    It uses also the console api.

    1. To do this will require us to rewrite the Console’s rendering pipeline. And its buffers (underway). And several other bits and pieces … but we’re getting closer 😉

  15. All of this is …nice, but when is the native, tabbed console coming ? Thats the true feature that everyone keeps asking whenever one raises the topic of windows console improvements.

    1. While Tab support is a common request, it’s FAR from the only one.

      Alas, before we get to tab support, we have to do a ton of re-engineering of the Console’s internals. We’re progressing well with this work and hope to turn our attentions towards UX features “soon”.

    2. ϻοϰsϯεr says:

      Personally, I’m happy about the changes and believe a proper terminal with good color support is worth much more than tabs. I’m happy they seem to prioritise terminal improvements over tabs.

  16. My humble $0.02 -> Instead of doing this kind of mish-mash here and there to be like *NIX; why not re-acquire XENIX, make it up-to-date and put Windows* as its Graphical Desktop System.

    1. why not re-acquire XENIX, make it up-to-date and put Windows as its Graphical Desktop System?

      because I think you wildly underestimate the complexity and significant differences between how Windows and *NIX-based Operating Systems work. Doing this would be like trying to mate a Tesla chassis to a Ford F150 engine, transmission, and running gear.

  17. v fatica says:

    Build 18198
    Win-R … cmd … open “Properties” dialog … do nothing … press “OK” … back to console … colors have changed from 7 on 0 to 0 on 0 (hard to see!). And HKCU\Console\%SystemRoot%_system32_cmd.exe\ScreenColors has changed from 0x00000007 to 0x00000000.

  18. KiYugadgeter says:

    Colors support are great!!
    But there is no information about Font support.

    1. What information are you looking for?

  19. ϻοϰsϯεr says:

    Hi Rich,

    great news. There is a thing that bothers me from some time know. It’s not exactly related to this blog post but it’s about the settings. And since you are already working hard on that, here is some feedback from me.

    I open the powershell in many different ways. Sometimes I click on the PowerShell icon on the TaskBar or on the start menu. Sometimes I press WIN-R and type powershell into it and sometimes I SHIFT-RIGHTCLICK on a folder and choose “Open Powershell here…”.

    The biggest issue I have with the PowerShell (or the cmd) is that each way opens a PowerShell with its own settings. So I have to set the font and the colors and whatnot to what I like at least 4 times. Then there are also the default settings, but they don’t overrule or overwrite my settings anywhere, so I don’t really see the point in that.

    This is because — as you well know — the settings are saved in the shortcut (somehow), right? So when I create five shortcuts to the powershell.exe on my desktop, each has it’s own settings.

    I’d love if I could tell the powershell to use the same settings profile-wide so that it wouldn’t matter what shortcut I’d use to open the powershell and I always have the same color scheme, font size, terminal size and so on… That would greatly streamline my experience with powershell.exe or cmd.exe.

    Hopefully you could share your thoughts on this with me.

    1. This post should provide you some clarity: https://blogs.msdn.microsoft.com/commandline/2017/06/20/understanding-windows-console-host-settings/

      When you run PowerShell via the Start Menu icon, the new PowerShell instance is invoked using settings stored in the tile’s shortcut. Same for PowerShell pinned to your taskbar – which gets its own separate shortcut and thus separate settings. When you start `powershell.exe` from the run dialog, there’s no shortcut involved, so PowerShell gets its settings from the Registry.

      Alas, this is impossible to change at this point since SO MANY apps, tools, scripts, etc. depend on this many-decades-old behavior.

      However, we agree that its’ confusing and far from optimal and are keen to improve it … eventually. Stay tuned!

  20. jimdeb3 says:

    please just take wsltty from mintty (https://github.com/mintty/wsltty), it’s already a fully featured linux terminal

    1. That won’t solve the Windows Console related issues. We are working towards a plan wherein we modernize the Console experience for everyone.

  21. AlexAegis says:

    Can we get an option to have a blurred and transparent frosted-glass effect on the console background to match the fluent design? 🙂

    1. It’s on the backlog. We’d love to make the Console more Fluent, but have some important work ahead of it in the queue. Bear with us 😉

  22. ϻοϰsϯεr says:

    By the way, the Terminal tab in my German OS is all over the place. Looks like a whirlwind went over the UI elements.

Skip to main content