Why aren’t console windows themed on Windows XP?


Commenter Andrej Budja asks why cmd.exe is not themed in Windows XP. (This question was repeated by Serge Wautier, proving that nobody checks whether their suggestion has already been submitted before adding their own. It was also asked by a commenter who goes by the name "S", and then repeated again just a few hours later, which proves again that nobody reads the comments either.) Knowledge Base article 306509 explains that this behavior exists because the command prompt window (like all console windows) is run under the ClientServer Runtime System (CSRSS), and CSRSS cannot be themed.

But why can't CSRSS be themed?

CSRSS runs as a system service, so any code that runs as part of CSRSS creates potential for mass havoc. The slightest mis-step could crash CSRSS, and with it the entire system. The CSRSS team decided that they didn't want to take the risk of allowing the theme code to run in their process, so they disabled theming for console windows. (There's also an architectural reason why CSRSS cannot use the theming services: CSRSS runs as a subsystem, and the user interface theming services assume that they're running as part of a Win32 program.)

In Windows Vista, the window frame is drawn by the desktop window manager, which means that your console windows on Vista get the glassy frame just like other windows. But if you take a closer look, you will see that CSRSS itself doesn't use themed windows: Notice that the scroll bars retain the classic look.

The window manager giveth and the window manager taketh away, for at the same time console windows gained the glassy frame, they also lost drag and drop. You used to be able to drag a file out of Explorer and drop it onto a command prompt, but if you try that in Windows Vista, nothing happens. This is a consequence of tighter security around the delivery of messages from a process running at lower integrity to one running at a higher integrity level (see UIPI). Since CSRSS is a system process, it runs at very high security level and won't let any random program (like Explorer) send it messages, such as the ones used to mediate OLE drag and drop. You'll see the same thing if you log on as a restricted administrator and then kick off an elevated copy of Notepad. You won't be able to drag a file out of Explorer and drop it onto Notepad, for the same reason.

Comments (124)
  1. osexpert says:

    drag & drop into cmd is a great loss and its one of the most annoying things in Vista. Vista is so protective with its high security, protecting the user from getting the work done. Nice work.

  2. CGomez says:

    No, I’d much rather go back to when users were in charge of protecting themselves and running as non-admin.

    I mean, back then, MSFT took all kinds of blame for every virus and trojan horse known to man, and it spawned a set of Mac commercials that routinely stretch the truth to the breaking point of lying.

    But at least then I could drag a file onto the cmd window that I hardly ever use so the file name would show up.  In fact, I’m not sure I ever did it.  But things were better then!

    [Okay, so if somebody finds a security issue like “Dropping a carefully crafted data object onto a command prompt could allow local elevation of privilege”, I’ll say “Sure that’s a security vulnerability, but CGomez likes it that way.” -Raymond]
  3. Triangle says:

    You never know how much you have until it’s taken away.

    [Okay, so if somebody finds a security issue like “Dropping a carefully crafted data object onto a command prompt could allow local elevation of privilege”, I’ll say “Sure that’s a security vulnerability, but CGomez likes it that way.” -Raymond]

    No, that’s not what you’ll say. You’ll say “We’ll fix this bug in CSRSS right away!’, or ‘We’ll move the console handling code out of CSRSS right away!’.

    [The bug in CSRSS was allowing data object code to execute. And that’s what was fixed in Vista. -Raymond]
  4. A followup question that may be on many peoples’ minds is — why exactly does the console have to run under CSRSS, when other shell-like program-launching tools (like, oh, the shell/Explorer) don’t need to? Essentially, what’s different about a console that requires additional services from a high-security subsystem like CSRSS, necessitating all these limitations.

    Bummer about drag & drop to the CMD prompt. That would be a real loss in functionality. Maybe third-party command prompt tools will make up for this, or perhaps a service pack could provide a registry setting for daring and responsible users to re-enable it.

    [If you did that it wouldn’t be a console subsystem any more now would it. Okay, fine, consoles run unprivileged. Now any program can kill the console system. That doesn’t sound good either. (The window manager runs as privileged code after all and nobody complains about that.) But I didn’t design it and I wasn’t there when it was designed and I haven’t even had lunch with the people who did design it, so I’m just speculating. I think I should stop speculating now. -Raymond]
  5. Sitten Spynne says:

    Would one be correct to assume that the reason why clipboard operations have to go through the console window’s menu is the reason discussed here?  I’ve been cursing the lack of CTRL + V into the command prompt for a while and if this is the case I can at least accept that it’s not technically a good idea.

    [It would suck if you couldn’t type Ctrl+V into any console programs, wouldn’t it? -Raymond]
  6. Richard Avila says:

    "restricted administrator" — what a lovely oxymoron. :)

    The whole concept of the user being an administrator in the first place is flawed but that’s off topic.

    • And yes, why does CMD need to run under CSRSS?

  7. grumpy says:

    So the (bastardized) "solution" is to run explorer as admin; I am only slightly joking.  You know, eventually Windows will be so secure that we won’t be able to actually do anything useful with it.

  8. Triangle says:

    [The bug in CSRSS was allowing data object code to execute. And that’s what was fixed in Vista. -Raymond]

    The real bug is that implementing copy and paste requires letting a data object execute code. Since Vista is all about improving security, how come that hasn’t been changed? It looks like Microsoft is trying to fix the wrong thing.

    [? Copy and paste works just fine in console windows. -Raymond]
  9. Jon Canady says:

    I don’t want to come off as a dick here or anything, but this is a standard theme (no pun) of Microsoft design.

    Instead of sticking it out and finding a good way to theme command prompt windows while allowing messages like Drag and Drop to pass through securely, you hide behind the “that’s just how it is.” excuse.

    Also, as Xenon already asked: why the devil do your command prompts have to be run by a special subsystem.  All decent operating systems don’t have such crazy requirements about their terminal windows.

    [GUI apps run under their own subsystem, too. But nobody complains about that. Go figure. -Raymond]
  10. he who is not him says:

    He didn’t design it, he wasn’t there when it was designed, and he hasn’t even had lunch with the people who did design it.

  11. Triangle says:

    "He didn’t design it, he wasn’t there when it was designed, and he hasn’t even had lunch with the people who did design it."

    Nobody said he did. You’re misunderstanding the question, which goes along the lines of "Why are you defending this obviously broken design, and why is more effort being put into working around it than fixing it?"

  12. Triangle says:

    [? Copy and paste works just fine in console windows. -Raymond]

    I meant drag & drop. Typo, sorry.

    [Think of that the next time you drop an Excel chart into an email message. -Raymond]
  13. Jon Canady says:

    [GUI apps run under their own subsystem, too. But nobody complains about that. Go figure. -Raymond]

    Nobody is complaining that CMD windows run under their own subsystem.  Nobody cares.  What people are complaining about is that (in XP) console windows aren’t themed or (in Vista) console windows don’t accept Drag and Drop messages.  

    The reasons given, however technically valid, seem like a cop-out.  

    And I still have no understanding of why CMD windows run in a different subsystem anyway.  That’s probably a much more involved topic than befits a blog comment field though.

    (This is not flamebait or anything, I’m not trying to start any Internet Drama, I’m just a curious use.)

    [It’s called “defense in depth”. Sure there may not be something wrong today, but somebody might discover a clever attack tomorrow. Better not to leave open the opportunity. I find it fascinating that people hate it when Windows takes a defensive security posture while at the same time lambasting it for having taken a more trusting security posture in the past. -Raymond]
  14. CGomez says:

    Raymond, I humbly apologize for not making the sarcasm drip enough in my comment.  I just could not believe the words posted by osexpert.  It just reeked of a "things were better when we had wood burnin’ stoves" argument.  I do not have one word of disagreement with your post.

  15. Eric says:

    [GUI apps run under their own subsystem, too. But nobody complains about that. Go figure. -Raymond]

    Yes, but that doesn’t create interference since we (almost) only interact with GUI programs. The console is in many ways a GUI to the command interpreter and whatever magic emulates a DOS box. Why not have the message go to the GUI, and GUI "types" it in ?

  16. Jon Canady says:

    [It’s called “defense in depth”. Sure there may not be something wrong today, but somebody might discover a clever attack tomorrow. Better not to leave open the opportunity. I find it fascinating that people hate it when Windows takes a defensive security posture while at the same time lambasting it for having taken a more trusting security posture in the past. -Raymond]

    This explains nothing.

    Why are console windows treated differently?  I understand that they are run at a higher security level and that tighter security is required, but nobody knows why the console is run at the higher security in the first place.

    And again, the issue of usability is ignored.  Could nobody come up with a way to determine if an object dropped on a command window is a file on a local or remote filesystem and if so insert the local/UNC path?  Forgive my ignorance, I’ve never coded any OLE drag-and-drop stuff, but if simply checking the type of a dropped object and possibly generating a path to a file involves letting the object execute custom code, that seems a little brokeish.  (Of course, I still don’t know why CMD windows have special privileges in the first place.  If they didn’t the whole issue would be moot.)

    (Again, if this comes off as personally offensive please do not take it that way, this is not my intent.)

    [Please read up on OLE data transfer. The way to figure out what something is is to ask it, which is itself a risky operation. Console windows are all owned by the console subsystem, and since it’s a subsystem, it runs with high privileges. Could this be changed? Beats me. -Raymond]
  17. Jon Canady says:

    In case I wasn’t clear — Microsoft taking a more defensive security posture is great.  There’s nothing inherently wrong with it.  But when it gets in the users way, it’s not being done right.

    For example, if installing an application as an unprivileged user takes anything more than typing my (or an administrator’s) password into a dialog, it’s broken.  If certain types of windows run under different privilege levels, that’s fine.  If I can’t drag things into that window, though, it’s broken.  

    (Yes, I’m aware that the “drag and drop a file into a console window” example is trivial, but it’s highly indicative of the type of decisions that seem to go into Microsoft products.)

    [Low-security processes used to be able to send messages to high-security windows. It was declared a security vulnerability. See “shatter attack”. -Raymond]
  18. Tom says:

    I’ve missed being able to drag and drop files onto elevated Visual Studio instances on Vista.  It’s probably my biggest incentive to not run Visual Studio elevated.  A lot of stuff in Visual Studio works without requiring elevation, but not everything.

    I suppose, based on my last experience commenting here, there could be a checkbox in the group policy editor somewhere saying "don’t screw with my drag and drop!" :)

  19. David Brooks says:

    What’s more, console windows running under Vista Basic seem to have acquired an extra-thick and squared-off window border that reminds me of the worst pixel-hungry excesses of OSF/Motif (which I helped design).

    Just sayin’.

  20. Heh. At least now I know that I gnash my teeth at intentional behavior. ;) It’s just too bad that the context menu doesn’t have a "Copy full path" item… That’d resolve the current many clicks to get something useable. (Of course, if things would stop using GUIDs, I wouldn’t have to worry quite so much about typos! ;) )

    Maybe I’ll add such an item to my registry, now that I think about it… It would actually be more generally useful than drag-n-drop, anyhow. (Example: You’re editing a text document in Notepad. You decide to include the default path to a file. You drag, then drop. "Oops…")

  21. DriverDude says:

    Thank you, Raymond, for explaining why colsone drag-drop stopped working in Vista! (Now why couldn’t the shell team explain this on their own blog when people complained last year…)

    And thank you, MSFT, for actually doing something to improve security. I thought that "Vista is the most secure OS" line was just marketing, but I’m surprised things actually changed under the hood.

    Of course I also want drag-n-drop into the cmd.exe, so now I’ve gotta figure out something that doesn’t involve CRSS…

    "You know, eventually Windows will be so secure that we won’t be able to actually do anything useful with it."

    No, it will be so secure that it will be just like Unix. I say that half-jokingly, because the last time I checked, I couldn’t drag a file into a emulated DOS program either.

    Being able to run a DOS program from the ’90s is a good thing. But being able to "trust" that a closed-source & propritary OS is reliable/secure out-of-the-box, is a much greater value.

  22. Gah. How’d I miss your Windows Confidential on "Explorer Doesn’t Do Text"? One of the best things about Vista is that it seems almost everything that’s been "taken away" has been replaced by somethig "new and shiny". And I’m not one to complain about new and shiny! (If it works…)

    (For others who missed Raymond’s column, shift-right-click a file. See that new "Copy as Path"? That’s far more useful than drag-and-drop, in general, no? :D )

    Regarding the subsystems, I believe the POSIX subsystem still quite exists in Vista. And the OS/2 subsystem started getting dismantled in 2000. (Though you could mount HPFS volumes if you wanted to take some files from NT 4…) I’m sure MSDN has a very good list of subsystems and their features, somewhere…

  23. @Matt Lewandowky:  You want "Copy full path"?  You’ve got it!  Hold down the Shift key while right-clicking on a file or folder object in the right pane.  See "Copy as Path" appear in the context menu.  And if it’s a folder, you’ll also get "Open Command Window Here".  (Why doesn’t it appear by default?  My guess is that it’s to avoid creating menu clutter for most users, but still make the options easily available for power users.)

  24. Keith Patrick says:

    Is the same true of PowerShell? (I don’t have a machine running it handy right now, so I can’t check)

  25. @Keith Patrick:  Yes, it’s true of any console windows – they’re all owned by csrss.

  26. Grant says:

    [Instead of sticking it out and finding a good way to theme command prompt windows while allowing messages like Drag and Drop to pass through securely, you hide behind the "that’s just how it is." excuse.]

    I find it hilarious that people accuse Microsoft of not working hard enough to meet users’ needs, when at the same time, others accuse Microsoft of working too hard to meet users’ needs (for example, the amount of work that goes into ensuring 12 year old Windows apps work in Windows Vista).

    If Microsoft devoted X person-hours to finding a way to safely theme the console window, someone else would ask why Microsoft didn’t spend those X person-hours fixing some other problem they perceive to be the biggest problem they encounter instead of making the console window look fancy, which is "just fluff".

  27. Martin says:

    [I find it fascinating that people hate it when Windows takes a defensive security posture while at the same time lambasting it for having taken a more trusting security posture in the past.]

    Really? Fascinating? Which part of it is that you find fascinating? That people critize your because it takes long time to add basic security mechanisms, or because users get annoyed that they need to re-learn how your system works because you didn’t get it right when you did it the first time?

    [Apparently, the only acceptable solution in your mind is to invent a time machine. -Raymond]
  28. Triangle says:

    If Microsoft devoted X person-hours to finding a way to safely theme the console window, someone else would ask why Microsoft didn’t spend those X person-hours fixing some other problem they perceive to be the biggest problem they encounter instead of making the console window look fancy, which is "just fluff".

    In case you haven’t noticed: This blog entry was about console windows. Complaining about a different problem would be off topic.

  29. ccutrer says:

    I don’t get it.  CSRSS = Client/Server Runtime SubSystem… a.k.a. Win32 subsystem.  There are several subsystems in windows, but few are actually used.  lsass.exe (Local Security Authority System Service) and lsm.exe (Local Session Manager Service) run under the native API (i.e. they link to ntdll.dll, instead of kernel32.dll).  There’s also the POSIX subsystem and the OS/2 subsystem that are very rarely used.  CSRSS/Win32 is the subsystem used by 99.9% of all Windows applications, and essentially encompasses linking against kernel32.dll (always loaded immediately after ntdll.dll after starting a win32 process), user32.dll and gdi32.dll, as well as advapi32.dll.  The actual csrss.exe process is run as SYSTEM, one process per session, and all other win32 processes communicate with the csrss.exe using LPC (Local Procedure Call).  The csrss.exe process hosts GDI and USER objects (windows, menus, atom tables, brushes, device contexts, etc.).  Some functionality is implemented in win32k.sys (win32 kernel mode).      

  30. Martin says:

    [Apparently, the only acceptable solution in your mind is to invent a time machine. -Raymond]

    Sorry, you got it wrong – I have never complained about Microsoft forcing me to re-learn stuff. I think most of the changes you make are to the better and improving your software. I almost can’t wait to download the new Windows Vista version torrent.

    What’s I described as fascinating (you’ll see if you re-read my post) is that you find the “complaining user”-behavior fascinating. You’re dealing with people. People complain if they have to do an extra mouse click, regardless of reason. End of fascination.

    [Well, yeah, people aren’t rational. But it surprises me how that irrationality is manifested. How people complain about exactly the thing they demanded, and then proceed to justify the complaining with “Well the only way to make me stop complaining is to invent time travel.” -Raymond]
  31. Foolhardy says:

    Back when NT 3.1 was designed, the idea was that the OS would support multiple APIs, i.e. Win32, OS/2, and POSIX in a modular way. Each of these environment subsystems had a server process running as SYSTEM to coordinate the environment: csrss.exe, os2ss.exe and psxss.exe, respectively. These are launched by the session manager. See HKLMSYSTEMCurrentControlSetControlSession ManagerSubSystems

    csrss seems to be a modular host for subsystem  servers, loading DLLs to do most of the work. When the session manager launches csrss, it tells it to load basesrv.dll and winsrv.dll. winsrv.dll is called twice: UserServerDllInitialization (the window manager) and ConServerDllInitialization (the console server).

    A process uses a console window by sending LPC messages to CSR.

    I think some of the features of having a common, privileged console server is that it allows multiple processes to share a console window, even if they’re running at different privilege levels. SYSTEM integrity the only level that everyone can trust. Also, it makes it easier for a non-win32 GUI process to have a console window, since you don’t need to load user32.dll or be a full Win32 client if another process owns the actual window.

  32. Triangle says:

    People complain if they have to do an extra mouse click, regardless of reason. End of fascination.

    http://en.wikipedia.org/wiki/Muscle_memory

  33. aargh says:

    I think the reason people are complaining is that other OSes don’t have a console subsystem, so the only way to make a console window is to manually code it with windowing APIs. That’s why other consoles don’t let you switch the video card into text mode and back (Alt+Enter), let every program have its own command history, or have per-exe macros.

    If Microsoft were to go and implement an xterm-style console, they would lose all the other features. Then everybody would say that they don’t care about drag-drop filenames, they just want their command history back.

    So those that want to be able to execute arbitrary code in the console program are free to implement it themselves, or just use cygwin’s xterm. Everybody else will have to live with using the clipboard.

  34. Dean Harding says:

    @Aaron Margosis: I understand that is changing, though… It’s only for version 1.0.

    One of the things people are complaining about is that cmd.exe (which was introduced in NT 3.1) has some design flaws and that changing those design flaws would essentially require a re-write. PowerShell IS a rewrite, and so decoupling it from (legacy) cmd.exe would seem like a good move.

  35. anonymous says:

    [Apparently, the only acceptable solution in your mind is to invent a time machine. -Raymond]

    Even that wouldn’t necessarily work, as it is commonly theorized that a time machine would not be able to travel further back in time than when it was built.

  36. ccutrer says:

    Ah yes… AttachConsole API function.  So I suppose that the console object is owned by CSRSS.  Multiple processes (cmd.exe) can share consoles (try having a Win32 GUI app attach to it’s parent console, and then write output to it;  it gets interleaved with the output from cmd.exe).  In this sense, the console is simply the character input and output.  The commands are still processed by cmd.exe (or whatever sub-process is currently currently), presumably by using ReadConsoleInput API.  It all begins to make sense.

  37. Gabe says:

    The basic problem is that a process’ message loop has to run somewhere. For an ordinary Windows program, the message loop obviously runs within the program’s process. For a console program, the choice is to run the loop within the program’s process or CSRSS’s process. Since a console program might not be Win32, the message loop has to run within CSRSS.

    Of course you could then have a CSRSS console surrogate process, running as the console program’s users, whose job was to run the message loop. Implementing that would be a lot of work to just to get themed windows and drag-and-drop. Keep in mind that you would need a separate surrogate process for each user token that runs console programs, which would still prevent your non-admin programs from communicating with your admin programs, and such.

  38. Gazpacho says:

    "What defines these "different subsystems"? How are they different from each other?"

    http://www.google.com/search?q=%22nt+subsystem%22

    NT was originally supposed to be the core of 32-bit OS/2, but Bill Gates also wanted something that could be adapted if OS/2 didn’t work out, which it didn’t. That’s why we now have one OS that can run Win32, Win16, and DOS apps at the same time.

  39. Mihai says:

    “How people complain about exactly the thing they demanded”

    You are mixing things. People ask better security, and complain about loosing drag-and-drop. Not the same thing.

    It might be a technical reason why creating a folder in “Program Files” asks for permission 4 times. But from user perspective is still idiotic.

    When adding something breaks somethings else it means the system is brittle.

    [They may not look like the same thing but they are connected. -Raymond]
  40. Miral says:

    I’m a bit dubious about console handling in CSRSS anyway.  I was recently testing a console app in a VM and discovered that simply running the app could quite repeatably cause the entire VM to spontaneously reboot.

    Turned out the problem was with the Vista manifest (the VM was running unpatched XP SP2), which update KB926255 fixes.  But arguably the console windows ought to be isolated better than this, so that at worst the single console window affected will close without taking out the whole computer.

  41. jd2066 says:

    Reading this post makes me wonder how WindowBlinds could theme console windows in XP.

  42. JamesNT says:

    jd2066,

    Because WindowsBlinds breaks about every rule written regarding how to write software for Windows.

    JamesNT

  43. ChrisMcB says:

    I don’t get it. People want to use GUI concepts on a COMMAND PROMPT? A Commend Prompt is NOT a GUI.

  44. BOFH says:

    I didn’t even realize that any drag’n’drop-feature was gone in the Vista Command Prompt.

    The question is what it was doing there in the first place, the command prompt is by definition a keyboard input interface.

    I have just kept pressing alt+space,e,p,enter when I want to insert text from the clipboard into my command prompt, and that works the same in Vista, just as it should.

    An annoyance related to this, is the new "address bar" or whatever it’s called, in Vista Windows.

    It doesn’t show and automatically highlight the text of the path to that window anymore.

    Now when I want to add a long and cumbersome path into a command prompt, there is an additional step:

    1. Press alt+tab from the command prompt to the window in question.
    2. Press tab until focus is moved to the address bar.

    3. (new step) Press enter to change the field to show the actual path and automatically highlight the whole string.

    4. Press ctrl+c to copy it to the clipboard.

    5. Press alt+tab back to the command prompt.

    6. Press alt+space,e,p,enter to insert the path into the command prompt.

    Sounds like a lot of typing, you think.

    Onlookers just see a brief flickering of windows on the screen for a second when I do this, it’s fast and instinctive.

    Although now the additional step is forcing me to learn new tricks whenever I have to touch a Vista box.

    I usually work with Windows Server and don’t have Vista on any main work or home computers.

    If I can find a way to make the address bar default to the original view, I would be more willing to switch to Vista, it’s the last UI-annoyance I haven’t found a solution for.

    If anyone has a suggestion on how to accomplish this, I’d be glad to hear it.

  45. Consolas says:

    You may not be able to use Ctrl+V to paste into a Command Prompt but you can still paste using only the keyboard, it just takes a couple more keystrokes: Alt+Space E P.

  46. Max says:

    BOFH,

    "It doesn’t show and automatically highlight the text of the path to that window anymore."

    Just hit Alt+d. It’ll focus on that address bar just like in IE and will highlight the whole string for you. ;)

    As much as I use keyboard shortcuts and hot keys I would hate to keep hitting tab or alt+tab to get to the address bar.

    Max

  47. jd2066 says:

    @JamesNT: Really? How do you know that? From what I’ve read Stardock is a certified Microsoft partner and WindowBlinds is Windows Logo Certified so it doesn’t seem like it could be breaking every rule on writing Windows software.

    [Logo certification doesn’t mean that it has been mathematically proven to follow every rule for writing Windows software. There are an infinite number of “dubious ideas” and only a finite number of logo rules. -Raymond]
  48. osexpert says:

    I see many say its ok to remove existing functionality when security is improved. If so, i hope MS removes IE, Messenger and ActiveX next. Most security problems solved.

    They should have fixed it the hard way by fixing csrss. This old relic from the microkernel wannabe days, where win32 once lived, deserve to die. So long old pal. RIP

  49. Karellen says:

    OK, this may be one of those questions that I should know the answer to before reading this blog, but I’m not sure so here goes.

    What defines these “different subsystems”? How are they different from each other?

    Is it like a fixed enumeration of possible subsystems in the kernel (e.g. console, gui) and every process gets assigned one? If so, what does the kernel subsequently *do* with this information? (How is it assigned? Is it defined by the executable type)

    Looking around msdn, I get a number of different hits for subsystem (disk subsystem, SUA, MAPI subsystem, Exchange event notification subsystem, print subsystem) that seem to be talking about a number of different types of thing.

    Links to a relevant MSDN (or other) page giving more details about what we’re actually talking about here gladly appreciated.

    [I don’t know either, but it is my recollection that there used to be more subsystems (POSIX, OS/2)… -Raymond]
  50. joel says:

    Someone above misunderstood how terminals work on Unix (at least modern Unix).  It’s not that different from Windows, but it allows ANY process to be the equivalent of CSRSS.EXE.   Basically, there is a file called /dev/ptmx (the pseudo-terminal master) which can be opened by a terminal emulator.  This operation creates a file in /dev/pts (usually), such as /dev/pts/0.  A program that wishes to use the terminal opens /dev/pts/0.  All communication occurs over the pipe between /dev/pts/0 and the terminal master (stdin and stdout).  Additionally, ioctls can be used to modify the workings of the terminal.

    The nice feature of this system is that anybody can create a console and connect any number of programs to it (by manipulating their stdin and stdout file descriptors to point to /dev/pts/X).  Consoles can be contained with consoles, such as is used with SCREEN.  By changing permissions on the /dev/pts/X files, you can control access on a user/group basis to the terminal.

    [And to read from the screen buffer you… oh wait, you can’t read from the screen buffer. -Raymond]
  51. BOFH says:

    > Just hit Alt+d. It’ll focus on that address bar just like in IE and will highlight the whole string for you. ;)

    Ah yes, much better than the stupid F4, and it works in WinXP too.

    But still a deviation from my finely honed, current practise…

    > As much as I use keyboard shortcuts and hot keys I would hate to keep hitting tab or alt+tab to get to the address bar.

    It’s actually just one tab in XP to switch to the address bar, and hitting tab is easier than moving my thumb to Alt and simultaneously reaching for D. :)

    Well, I guess I’ll have to learn some new fastfinger-routines eventually, and this Alt+D looks like a good alternative, since it works in current Windows Server and XP too.

    I couldn’t handle having different keyboard shortcuts in different editions of Windows, so that seems like a good compromise.

    [I can’t believe anybody used anything other than Alt+D to get to the address bar. That’s why the D is underlined in “Address”… -Raymond]
  52. aargh,

    You must not use Vista much, else you’d be familiar with "This system does not support fullscreen mode" by now. ;)

    But you brought up the core of the issue: Windows != X. Oddly, I never seem to *need* to drag/drop onto console windows in OS/2 (never thought about why, but now I’m curious as to how my usage pattern differs…), so I don’t know what its behavior is off the top of my head. (OS/2 would really be the only fair comparison here, considering the shared, sordid, history.)

    But, as I said/implied/whatever, in an earlier comment: I prefer being able to get the path easily over being able to drag/drop the icon. After all, dragging an icon into an app usually implies that you want the app to "open" the file/folder. And that is most definitely not what one wants their command interpreter to do with $randomFile, as a rule…

    So, in this case, Microsoft’s done the "Right Thing" by not ony removing drag and drop into consoles, but by adding a new (if obscure) feature which is actually more powerful overall.

    (If only I could have a full-screen command prompt again. And if only mode.com allowed one to set the number of lines, still…)

  53. josh says:

    [It would suck if you couldn’t type Ctrl+V into any console programs, wouldn’t it? -Raymond]

    That would happen if Ctrl+V was implemented at the console layer, but you could still get Ctrl+V at the command prompt by implementing it in cmd.exe.  It wouldn’t work for all console-mode applications, but that’s how it works for windowed apps anyway.

    But I guess it might be important *somewhere* to be able to enter character code 22 on the command line someplace.

    [Are you willing to give up command history for this feature? (Command history uses line mode, but reacting to Ctrl+V at the time of entry can’t be done in line mode; it would require character mode.) -Raymond]
  54. Nathan says:

    I have a question..

    Why doesn’t the microsoft symbol server include symbols for ALL versions of dlls. It seems to only have them for RETAIL builds.

    Nathan

  55. Gazpacho says:

    "But I guess it might be important *somewhere* to be able to enter character code 22 on the command line someplace."

    Only in little-used programs like emacs ;)

    You can’t seriously believe that "that’s how it works for windowed apps" is an acceptable answer for programs that, by definition, want nothing to do with windowing.

    (and we haven’t even mentioned the problems of not being able to type ctrl-C)

  56. Pim says:

    Most of the postings on this blog remind me of Douglas Adams’ “story of reason.” The parallel here is that there are always very good reasons behind every lamentable behavior in Windows, reasons that sometimes span back decades or can be linked to earlier criticisms. It is somewhat stunning to see how a chain of completely sensible decisions can, over time, lead to ludicrous results.

    Pointing back to these decisions sounds like a tempting way to defend the current state of affairs, but it carries the presumption that each of those decisions was the only correct one to make. Since there are (and have been) plenty of desktops that implement a command line window that accepts DND without running into even theoretical security issues, this presumption cannot be held true.

    What this boils down to is, I think, the level of conservatism that has been upheld over the years building Windows. I can appreciate the “damned if we don’t, damned if we do” scenarios you have to deal with when juggling a large installed base, but the choice between ‘backwards compatible’ and ‘completely changing everything’ is never binary. Assuming that it is so leads to shipping a command prompt that can still run lode runner but can’t accept drag&drop.

    Happy new year!

    [Drag and drop is built out of many small pieces. At the top is the console window, but underneath that is OLE, and underneath that is COM, and underneath that is marshalling… Sure the console might say “Okay, screw it, this is out of control. We’re going to use a different drag and drop model that doesn’t have as many risks” but how do you get all other programs to switch to the new model? Everybody thinks that the console gets to decide how drag and drop works (and therefore can redesign drag and drop to avoid whatever problems it’s worried about) but in fact it’s the top of a very complicated food chain. -Raymond]
  57. Pim says:

    [And to read from the screen buffer you… oh wait, you can’t read from the screen buffer. -Raymond]

    That’s an implementation issue. It’s not hard to imagine a Unix console device that would allow reading back the screen. Nobody bothered because unix console software sort of builds on the assumption that it’s talking to a dumb terminal. Programs that have a valid need for knowing what is on the screen tend to do this by remembering what they printed to it. Added advantage is that this concept also works sanely over a serial line (or an interactive network connection), pray tell how do you read from the screen buffer on the other side of a serial link?:).

  58. Joel, while your analysis is (somewhat) correct, it’s not really relevant. The appropriate "Unixy" equivalent to a "Command Prompt window" would be an "xterm".

    Since one cannot drag-and-drop files into an xterm (I’m ignoring Konsole/gnome-terminal/et al, as they are not directly comparable), the argument could be made that there is now "feature-parity" between platforms. Indeed, losing drag/drop to gain "Copy as Path", may well be worth it, as the former Command Prompt behavior differed from practically every other Windows application.

    Since the Command Prompt can’t "open" a file, the drag-n-drop metaphor is broken to start with. It was simply a convenience for those of us who really needed something like "Copy as Path". Now that we have "Copy as Path", I see little reason to mourn the loss of something which doesn’t make much sense when analyzed.

    BOFH, Alt-D ought to work on 9x, as well… And NT 4 if you manage to get Active Desktop installed. ;) It is generally better to rely upon "official" shortcuts rather than trying to remember the tab order in each OS revision. Less thinking, more speed. I’m sure that once you’re used to it, Alt-D will be faster than thinking "Which OS am I on again?" :)

    Osexpert, that comment’s irrelevant. Why mourn the loss of functionality which was essentially "broken"? Especially when we can celebrate that security enhancements made it finally disappear? And, by your reasoning presented, Microsoft may as well just pull executables from the OS altogether. IMO, locking down the environment’s saner than ripping things out wholesale. (Guess what Microsoft’s done with Vista? Hint: it takes specific elevation and confirmation to install a new ActiveX control…)

    And, RE: microkernels, Windows is still a "macrokernel", even though the importance of the Win32 subsystem increased in NT 4… At least they’re starting to emphasize the POSIX subsystem again… (The UNIX subsystem is really the POSIX subsystem, right? I’m not running Ultimate on this machine, to check…)

    I know Raymond’s much more capable than I am of describing these things. But I felt like responding. ;)

  59. aargh says:

    Matt Lewandowsky: I don’t use Vista much, but the very first time I used it, I did get a full-screen console. It’s a question of the graphics driver — some can do it, some can’t.

    osexpert: How do you propose "fixing" csrss?

    joel: Consoles (the UI element, not the OS construct) are completely different on Unix than they are on Windows. On Unix, some process (like xterm) acts as a terminal emulator and draws the window. It sends mouse commands and keystrokes to the child program via the pty hooked up to the child’s stdin. The child sends cursor positioning and color change commands along with output on stdout which comes in to the emulator’s pty. On Windows, CSRSS draws all the console windows. There are APIs which do things like get mouse events and set cursor position, which are separate from stdin and stdout.

    These fundamental differences manifest like this: If a Win32 program wants to open a console window, it calls AllocConsole() or AttachConsole(). A Unix program has to be attached to a console when it’s started in order to get its stdout displayed. If a Win32 program in the middle of a pipeline wants to change the window title (perhaps to report progress), it calls SetConsoleTitle(). A Unix program must send control codes to stdout to change the window title, but a process in the middle of a pipeline does not have its output going to the terminal emulator, so it cannot change the title.

  60. Pim says:

    Matt, it’s not hard to imagine a way to implement drag&drop in an event-based windowing system in a way that the terminal window can do something sensible with it. I have no idea how windows does it internally, but the sanest approach would be to implement a DROP event that includes a ‘content-type’ tag. UI regions that can receive drops can then set some properties to indicate what kind of content they accept. Within that set-up, dropping a file would trigger an event with a ‘url’ content-type. The terminal window could then handle drops of this type by sending the path as keyboard events to the terminal layer. It wouldn’t surprise me if this is exactly what happens if you drag a file to, say, gnome-terminal on a Linux desktop.

    Now I’m sure that stuff like OLE complicated drag&drop under Windows, but surely the basic scenario of dragging a file wouldn’t need to be so complicated?

    [So there should be two types of drag and drop, one for files and one for everything else? OLE data objects already have “content types” (called “formats”), so it already does what you want. The issue is that it also does more, and it’s the “more” that is risky. -Raymond]
  61. Raymond,

    Thanks for answering my question from the suggestion box.

    I don’t quite get the logic that backs up your rant about people not checking if their topic is already in the box.

    1. The Serge Wautier set doesn’t include the Everybody set.

    2. A double suggestion may also mean that the poster wasn’t able to find the first occurence of that suggestion. Which is the case here. I carefully re-did the exercice and failed again.

    You may conclude that people are stupid since they’re not able to find a related topic worded using different keywords in a list of 400+ posts (although I would send you back to #1 above).

    But I reject your conclusion about me being negligent. It’s just plain wrong. And it’s unfair to all people who do their homework, including those who are better than me at doing it.

    Happy New Year,

    [I argue that it’s my readers’ fault for failing to exercise restraint in posting suggestions that has led to the 400+ entry suggestion box. That’s why I closed the suggestion box. And yes, I know that Serge Wautier != everybody. It’s called rhetoric. -Raymond]
  62. aargh: AFAIK, no LDDM drivers support full-screen consoles. If you happen to be running an older "XP-compliant" driver, you can certainly get a full-screen console. But the expense is Vista-only functionality…

    Pim: But the point I was trying to make is that drag-and-drop of an icon into a program 99.999% of the time means, "Hey, Program! Open this file for me, would ya?" I think it’s broken for that 0.001% of the time to be your console. It’s counter-intuitive. Practically every other program that can’t "open" whatever file you drop onto it will simply silently refuse or pop up a dialog telling you so.

    Since a console window should not be actually opening files (on any platform!), Vista’s behavior is now consistent with the rest of the OS. Since Microsoft’s provided a way to achieve the previously-missing functionality that drag-n-drop worked around, I welcome the change.

    No more ‘echo <drag-n-drop file with obnoxious path> > %TEMP%1234.txt’ and opening the file in a buffer and copy/pasting that into the text document I’m editing, for example. Sadly, that is faster than the GUI alternative of showing the file’s location, copying that into my editor, then copying the filename into my editor.

    Generally, I need a file’s path in more than just a command prompt. So, the new way of doing it is far more flexible, and welcomed by me.

    Also, as mentioned earlier, your approach still requires "opening" the file to dicover what sort of content it is. I personally want to trust my console windows, so I’m not all that big on them accessing arbitrary files directly (or even indirectly…). That’s what the interpreter’s for, not the console itself.

    And just because GNOME does something doesn’t make it right. (See "they are not directy comparable"…) I could point out dozens of things they do "wrong", and I try to only use GNOME on Solaris so my experience is purposely limited… (And even then, I try to do everything under CDE…)

  63. Pim says:

    Matt, my point wasn’t that Gnome was doing everything right. I haven’t used it in ages, X11 depresses me. I was just laying out a way you could implement drag&drop of files that would allow any program to decide for itself what the ‘proper thing’ would be. In the case of file drags, you are right that the type of the file itself also needs consideration. It would still make more sense to take that meta-data into consideration than to go for a model where you bluntly dump the binary contents of the file on the receiving application. To me, a more sensible interpretation of a drop-event would be "perform the most sensible action for the combination of object type, receiving application and drop area". That doesn’t always have to mean "open for editing".

    The fact that there is a workaround doesn’t make up for the loss of coherence introduced. In my experience, the usability of drag&drop is a direct function of the user’s success ratio in dragging things in the interface and having the expected thing happen. This is one area where the Mac has always had an edge over windows and X11. At the risk of making my own anecdotal experience here bigger than it is worth, on X11 and Windows I find myself rarely bothering with drag&drop precisely because I can’t predict whether it will work, whereas on Mac (classic or OSX) I use drag&drop without restraint.

  64. BOFH says:

    I can’t believe anybody used anything other than Alt+D to get to the address bar. That’s why the D is underlined in "Address"… -Raymond

    You’re right, it is.

    Unfortunately I came to use the current method from another perspective.

    I was and am a big tab user already, for example by pressing ctrl+esc,esc to put focus on the Start button, and from there cycle through the toolbars, start bar, notification area (pressing shift+F10 on anything needed to quit or change here), to the desktop, and back to the Start button again.

    And then also via the tab command line expansion, which I would manually enable in WinNT and Win2K, but luckily it has been enabled since then.

    Alt+[shift+]tab and ctrl+[shift+]tab is second nature since long ago.

    About six years ago I trained myself off from using the rather awkward alt+F4 to the much more ergonomically convenient ctrl+w instead.

    That took strangely enough a long while to get used to.

    On the other hand, I was trained on Windows 3 + 3.11 back in 1992, where a part of the final exam was a practical test where I had to operate Windows without a mouse through a series of scenarios, and those old ways have stuck with me ever since. :)

    I almost never use anything else than alt+space,x and alt+space,enter to switch windows to and from maximized view, for example.

    Call me old-fashioned if you like…

    Another annoyance in Vista is that pressing ctrl+esc,esc,shift+F10 no longer produces a right-click menu on the Start button.

    I use the method to quickly gain access to the personal- and the all users-profiles.

    The Start button is not even a button anymore in Vista, it’s some strange oblong that doesn’t give any indication at all if it has focus or not.

    And what’s with backspace no longer navigating the window back through the folder tree?

    Sorry that I’m getting way off-topic here, but all these details that have suddenly changed in Vista make you wonder if they are the result of careful study of focus groups, or just a bunch of young whippersnappers at MS let to roam free and change whatever they wanted in the product.

    I’m surprized that there still hasn’t been a TweakUI released for Vista, it’s sorely needed.

    Just the other day I noted that I needed the "rebuild icons" feature from TweakUI on my only Vista notebook, when I noticed that it was impossible to create a shortcut from a VPN-connection to the desktop and have the shortcut maintain its icon after a reboot.

    It reverts back to the old "no icon" icon from the Windows 3 Program Manager every time.

    Nostalgic perhaps, but certainly unwanted.

  65. jd2066 says:

    @Matt Lewandowsky:

    I’m ignoring Konsole/gnome-terminal/et al, as they are not directly comparable

    Just wondering, why not?

    just because GNOME does something doesn’t make it right.

    It’s not just gnome, Konsole supports DND and it’s a KDE app.

    everything under CDE…

    Do you mean KDE?

    @Raymond: Why do you respond inside the original comment instead of using a separate comment?

  66. Pim says:

    Matt, I’m also interested in your point about xterm being somehow different from gnome-terminal or Konsole. I didn’t actually see that statement the first time because it doesn’t seem to make sense. Both are applications that open a pseudo-terminal and emulate a terminal proper in X11. The only difference is that Konsole and gnome-terminal adhere to the Open Desktop standards set for things like drag-and-drop where xterm is an example program that came with X11 back when hippies were in vogue. Am I missing something?

  67. Gabe says:

    Pim, you have correctly surmised how dragging a file works. The difference is that it was decided for Vista that high-integrity windows would not accept input from low-integrity windows.

    That means your malware can’t send keystrokes saying "uninstall AntiVirus" to the elevated command prompt. Unfortunately it also means that Explorer can’t send a message saying that the user just dropped "C:filename.exe" to the command prompt. Afterall, what if the malware had infected Explorer and C:filename.exe was the command to uninstall your AV software?

    This is different from other implementations (like X) in that the console windows are handled by a high-integrity process in Windows and by a regular user process under X; and X doesn’t have a way of preventing an attacking process from sending malicious messages (like "rm -rf /") to unsuspecting terminal windows with root privs. Presumably X doesn’t have this restriction because such attacks aren’t common. We used to do this sort of thing in college to people whose xhosts were misconfigured.

    As I said before, all it would take is to run the console message loop inside a process that runs as the current user to get themed windows and drag-n-drop. I just don’t think it’s worth the effort for such trivial features, though.

  68. Pim says:

    Gabe,

    Whether it is or isn’t worth it is always a hard choice to make. I would personally err on the side of DND-coherence. But perhaps I don’t exactly understand the security implications here. Do DOS prompts run at a higher privilege level? Does that mean there’s no DOS prompt for non-administrators? In the context of X11 there would be little point in subverting a user xterm; if the user can delete a file a simple call to unlink() would be more straightforward, unless if the user could be enticed to have a root shell open.

    I somehow get the feeling that a large source of this problem is the fact that no real distinction has ever been made between "the application you use to run legacy DOS apps on" and "the command line interface to Windows".

  69. Mihai says:

    A happy new year!

    And sorry for my annoying comments.

    I love your blog, I really do. And I am ok when you try to defend MS, I really do.

    But don’t go overboard. Don’t serve your readers some stuff that you would also not take in a rational discussion.

  70. Gabe says:

    Pim, as was mentioned before, the command prompt program (cmd.exe) runs in a different process from the message loop (csrss.exe). This is much the same as in X11 where maybe your shell (csh) runs in a different process from your terminal emulator (xterm).

    The difference here is that CSRSS runs with system privileges, so programs that run as you (like Explorer) can’t talk to it. Why is the system designed that way? I don’t know, but it means that implementing d-n-d is a whole lot more trouble than it’s worth.

    You could ask similar questions about X: Why does my X server have to run as root? Why can’t I get a console to switch between windowed and fullscreen mode? Why does my screen become unlocked when my screensaver crashes? Why does my screensaver have to run as root? And the answer to them all is the same: it’s because of an architectural decision that’s too difficult to change.

  71. Pim says:

    To be fair, after giving Gabe’s explanation a bit of thought, I can see how from where we are now this choice could even be defended as the most sensible, at least when you get to the point of inter-application trust. If the current state of affairs is that you cannot trust, say, explorer.exe to not be compromised, listening to events can become a dangerous game. I’d still say that it is regrettable that the DOS prompt is not ‘just another windows shell’ in the same way that explorer.exe is, in as far as they are both interfaces to the same environment you would want them to be at the same privilege level, where damage to explorer.exe already carries the same risks (being able to start C:malware.exe) as those exposed by accepting keyboard events for "malwaren" in the DOS box.

    If it were purely to amuse people who like dropping files on the DOS window, fixing that part would not be worth the cost. But it seems to me that getting rid of this elevation scenario would lead to a better design. If I understand it correctly, the reason behind the elevated privilege level of the DOS box has all to do with DOS programs that form the 0.001% part of the spectrum of actual real world usage.

    The same goes for not binding Ctrl-V to paste: console programs that accept meta-keys are part of the 0.001%, the other 99.999% of the cases you just want to paste. This is even true on Unix (of course Mac users got off light here, since there are have never been console programs that used the ‘command’ meta key).

    It seems to me that Microsoft has a philosophy where they will choose for the 99.999% solution if it saves work, unless if the other 0.001% can be tied to the totem of backwards compatibility. I may be overgeneralizing, or perhaps this choice may even have been worth it, in a business sense kind of way. But I still think it is regrettable.

  72. Pim says:

    Gabe,

    Every architecture has skeletons in its closet I am sure. The older the foundations of a code base, the worse it gets. I’ve just been silently hoping for Windows to grow a pair at some point and toss some of the worse ideas to the side.

    The ‘X runs as root’ factor is something I recall as a longstanding issue that has been tackled from various sides for a long time. You can, in fact, run X11 as a mortal user on top of the linux frame buffer device. People are constantly experimenting with ways to layer up stuff like hardware acceleration in a way that makes sense both in terms of performance and security.

    I’m not saying this to pooh-pooh your very valid point that sometimes you’re stuck on a foundation of earlier work that isn’t worth the effort to rip out. It’s just that these legacy-related problems in Windows seem to always be related to such an interconnected cluster of other parts that the whole process inevitably leads to architectural inertia.

  73. luca says:

    I never user drag&drop in a cmd windows, so I don’t care about it. Security is more important, and I like the integrity levels.

  74. Gazpacho says:

    That’s off-topic, but I’ll take a stab:

    * The VC library symbols are uploaded with a tool that other groups in the company use.

    * That tool is hardcoded to use only retail symbols, so that someone won’t accidentally upload decompilable code.

  75. Joel says:

    “These fundamental differences manifest like this: If a Win32 program wants to open a console window, it calls AllocConsole() or AttachConsole(). A Unix program has to be attached to a console when it’s started in order to get its stdout displayed. If a Win32 program in the middle of a pipeline wants to change the window title (perhaps to report progress), it calls SetConsoleTitle(). A Unix program must send control codes to stdout to change the window title, but a process in the middle of a pipeline does not have its output going to the terminal emulator, so it cannot change the title.”

    Two things.  One, it’s always possible to redirect stdout in the middle of a program to a terminal.  At the very least, the program can exec() an xterm and send its output there…if it really needs to do something like this.  In practice, I can’t really think of a case where a program would need to do what you suggest.  If someone wants stdout from a graphical app, they would run it from a terminal (the standard practice as far as I’m aware).

    Secondly, it wouldn’t make sense for a program in the middle of a pipeline to be changing the console title because it might also be run from a script, which might not even be attached to a console at all and shouldn’t care in any case.  However, if you really wanted to change the title, you could get the name of the file representing the terminal (probably by walking down the pipeline by looking in /proc/XXX/fd for each process further down the pipeline than your own), and then send the commands directly into the file, instead of into stdout.  Again, why one would want to do that is beyond me, but it’s not impossible.

    Raymond: for virtual consoles, as opposed to pseudo-ttys, you can get the contents of the screen buffer by opening /dev/vcsX and /dev/vcsaX (the former is just character data, the latter is color and character data).

    Since most programs that would actually need access to the screen buffer use a library like ncurses, which has its own buffer, there’s really no need to be able to directly read what’s stored inside the terminal emulator.  It’s also faster to look inside a local copy instead of grabbing something that lives in the memory space of another process.  

    [Title change by a program in the middle of the pipeline: In order to show progress without disrupting stdout. This is popular in batch files for long-running processes. And you can’t consult a private screen buffer you didn’t create, e.g. if you want to save the screen contents when your program starts, do some fullscreen editing, and then restore it when you exit. I guess people on unix simply accept that some things cannot be done. Too bad people on Windows can’t do the same. -Raymond]
  76. anonymous says:

    You stilled missed to answer the most obvious question: Has this also been fixed on Windows XP?

    (After all, Windows Vista is an unusable disaster with many design-based security vulnerabilities.)

  77. osexpert says:

    args: I don’t have the answer how to fix csrss, but a radical idea from KJK::Hyperion comes to mind: http://www.reactos.org/pipermail/ros-kernel/2003-October/000168.html

  78. osexpert says:

    typo: reply to aargh, not args:-)

  79. Bobris says:

    I want to know why the windows console is so many lightyears behind when compared to one of any other OS…

  80. Triangle says:

    Why mourn the loss of functionality which was essentially "broken"?

    Because it isn’t just the console. Any evelated program can nolonger be D&D’ed with by any nonelevated program.

    X doesn’t have a way of preventing an attacking process from sending malicious messages (like "rm -rf /") to unsuspecting terminal windows with root privs.

    Are you sure about this? I seem to recall something about there being a member of the message struct that is set when it’s a real message, and cleared when it’s a fake message.

    AFAICT, Windows has nothing like this.

    UI regions that can receive drops can then set some properties to indicate what kind of content they accept. Within that set-up, dropping a file would trigger an event with a ‘url’ content-type.

    I personally think it might be a better idea to  directly pass the handle to the object instead of a URL/URI. Imagine the following scenario: Program A has a file drop target, and Program B has a file picker. But program B has different rights than program A, and it shows some files that A would be unable to access. The user drags one of those files into B, and gets an incomprehensible "Access denied" error.

    The terminal window could then handle drops of this type by sending the path as keyboard events to the terminal layer.

    No. This is exactly the security issue that was fixed in vista. The window manager has to control the entire operation, not the app.

  81. Mihai says:

    “They may not look like the same thing but they are connected.”

    They are connected. But not only they “not look like the same thing,” they really *are not* the same thing.

    You want a spa in your home. I come to install it, and in the process I break the door (good reason: it was not big enough to fit the spa).

    And if you complain, I can also say: “How people complain about exactly the thing they demanded”?

    The two are connected. But they are not the same. And they don’t look the same. This is definitely not what you asked for.

    I don’t care much about the loss of drag and drop. But you arguments bothers me the same way you are bothered by nitpicking.

    Read this again, and this time imagine you read  it in a random post, attacking MS:

    “How people complain about exactly the thing they demanded”?

    “They may not look like the same thing but they are connected.”

    Does it sound logical?

    [“How people complain about things that, at the end of the day, and perhaps in ways they may not realize, are connected.” How’s that, Mr. Nitpicker? Because the mechanism that drag and drop uses can potentially be used by bad guys to do other things. -Raymond]
  82. Directx says:

    Windows platform guts were by design insecure, developed on a very different perspective, mostly targeting Home PCs and easy-of-use (that’s what made Windows famous). Even Windows NT failed to change that design attitude, and Vista just made things worst trying to strength-secure an insecure architecture -“UAC” for example is the first to turn-off after a Vista installation -really pity!

    However, I still believe (or hope) that there must be a way to implement all the needed security balancing the benefits of the Windows architecture in heritage instead of crippling some nice system features.

  83. Joel says:

    "[Title change by a program in the middle of the pipeline: In order to show progress without disrupting stdout. This is popular in batch files for long-running processes."

    I think the reason this never shows up in Unix is that people don’t expect batch processes to require a console to run.  If you want status reports, that’s usually done via log files, which can then be parsed and manipulated from programs outside of the pipeline, as needed by the user.  I feel sure, however, that if there really was a strong need to change the console title from within a pipeline stage, an ioctl would have been introduced for such.

    "And you can’t consult a private screen buffer you didn’t create, e.g. if you want to save the screen contents when your program starts, do some fullscreen editing, and then restore it when you exit. I guess people on unix simply accept that some things cannot be done. Too bad people on Windows can’t do the same. -Raymond]"

    This is solved by having multiple screen pages.  When a program goes into fullscreen mode, the terminal saves the contents of line-mode.  Then, returning to line mode results in those contents being restored.  This is standard on most graphical terminals these days.

    I admit some Unix solutions are a little hacky, but so are some Windows solutions, as we’ve been discussing.  The reality is that by the time we had systems capable enough to offer advanced terminals, the computing world had already moved on to purely graphical interfaces.  Sure, Unix could have offered access to the screen buffer for all terminals, instead of just consoles, but ncurses already does all the work except for a few very small edge cases that you mention.

  84. Neil says:

    Joel, /dev/tty always represents the terminal, assuming you have one.

    Raymond, there are apparently xterm codes to save and restore the screen; <ESC>?1047h and <ESC>?1047l

  85. Gabe says:

    Pim, I do not think you understand correctly. You seem to be confused about the elevation happens. This has nothing to do with the command processor running elevated. It’s that the process drawing the window is elevated.

    Why is an elevated process drawing the window? No reason in particular, it’s just that the most convenient place to do the drawing is in an elevated process. This has an unfortunate side-effect of preventing the window from communicating with other windows.

    This has nothing to do with the age of the codebase, it’s just an artifact of the design. Microsoft could certainly redesign the system so it doesn’t suffer from this flaw, but then it would suffer from some other flaw(s) instead. Since the program is old enough to acquired millions of users, introducing other flaws would not be welcome to them.

  86. Mike Dimmick says:

    I suspect that the reason the console WindowProc runs as part of the highly-privileged CSRSS process is historical.

    Back in NT 3.1, graphics drivers ran in user mode, in the CSRSS process. It would therefore make some kind of sense, if you accept that the WindowProc should not run in the actual process that created the console (which you might not want to do, for security reasons – do you really want different console-mode processes to do IPC to the console-creating process to do their output?) that it should run in the same process as the graphics driver.

    I can also see that the idea of an arbitrary process being able to switch into full screen text mode might not be desirable either. That said, DirectX programs can of course switch into other full-screen graphics modes.

    When graphics drivers were shifted into kernel mode for NT 4.0, this decision wasn’t revisited. As is often the case, you want to avoid disturbing code that already works fine. Possibly moving the console support to also be a driver could have worked, but drivers being able to create windows is a massive, massive layering violation.

    As for the comparison with Unix and X, it’s simply that the process writing to the console (e.g. csh) is performing IPC with the xterm/other terminal emulator via a ‘pseudo-terminal’ device (xterm is then doing IPC/RPC to the X server!) The Unix model is built entirely around terminal devices because that’s what there was when Unix was invented for the PDP-11. X is a much later arrival and has to be backwards-compatible with what went before.

    A better approach for Windows would probably be to strip the console management code out of csrss and into a new program, say ‘console.exe’. The Win32 subsystem would launch that (running as the session user) the first time a console was required, and reuse that process for future consoles required in the same session. For UAC you might launch a new console.exe for the different UI privilege levels, with the appropriate permissions, to prevent shatter attacks.

  87. osexpert says:

    Mike: “Possibly moving the console support to also be a driver could have worked, but drivers being able to create windows is a massive, massive layering violation”

    No, the driver would not create windows. It would “only” handle console objects containing textformatting. The new cmd.exe would then create consoles and do IOIOCTLs agains the console handle. cmd.exe would draw the text and handle window messages and stuff. The only thing I’m concerned about is console textmemory “living” in kernel mode (with maximum screen buffering: as much as 190MB per console), but this may not be problem at all.

    [Oh great. “Hello world” doesn’t work because you don’t have a message pump. -Raymond]
  88. Dean Harding says:

    "There is no cmd.exe anywhere."

    Actually, I was thinking more along the lines of Mike Dimmick’s idea of a "console.exe" running as current user to replace the functions that csrss.exe does for consoles.

    I still say it’s not worth the bother of re-implementing it all anyway… just to get drag & drop!

  89. Triangle says:

    [What if there is no cmd.exe? E.g. you Start.Run a console program directly. There is no cmd.exe anywhere. Who runs the message pump? -Raymond]

    Just a guess: kernel32, since that’s where AllocConsole, AttachConsole, etc. all come from. Although that wouldn’t be as efficient as Mike Dimmicks’ suggestion.

  90. Gabe says:

    Mike made the same suggestion as I did, only he called it console.exe instead of just a surrogate process. The problem is that the surrogate for elevated consoles would still not be able to receive messages from Explorer. It would need a low-integrity surrogate-surrogate (console-low.exe) to communicate with regular processes like Explorer and send the data back to console.exe, which would interpret the input and send it to the program running in the console (like cmd.exe).

  91. Dean Harding says:

    "It would need a low-integrity surrogate-surrogate (console-low.exe)"

    I don’t think you’d want that. You can’t drag&drop onto any other elevated windows, so you wouldn’t expect it to be possible to elevated command prompts either.

  92. Pim says:

    [So there should be two types of drag and drop, one for files and one for everything else? OLE data objects already have "content types" (called "formats"), so it already does what you want. The issue is that it also does more, and it’s the "more" that is risky. -Raymond]

    It would still be the same event, only it would have a content-type of ‘url’. If I’m not mistaken that’s the way OSX and the open desktop standard deal with drops. X11 has many quirks and I’m not advocating OSX to be perfect, but where it concerns drag&drop you can’t get much shinier.

    It makes sense to distinguish dropped files (that have a permanent presence outside of RAM) and dropped content out of other applications (that is temporary in nature). At very least I wonder what effect the idea of always sending the full content to the other side has when you drop a 2TB file on an application?

    I do realize that having a certain way of handling drag&drop is something that is tricky to change after the fact.

  93. Pim says:

    [Pim, I do not think you understand correctly. You seem to be confused about the elevation happens. This has nothing to do with the command processor running elevated. It’s that the process drawing the window is elevated.]

    Ok, at least that makes a bit more sense, but not in the context of the examples I was given (dropping “uninstall antivirusn” on the dos box), the presumption here was that dropping was dangerous because the shell had privileges. So now I understand the elevation but I no longer understand the vulnerability.

    [The drop event goes to the frame window. The frame window receives the dropped object from OLE and asks it for its “hdrop” content type (hdrop = “list of file names”). The dropped object is an object in the object-oriented sense (data + code), and it’s the code that’s the scary part. I’ll make a note to cover data object code-behind later this year. Maybe that’ll help.  I think people fail to understand how general-purpose drag/drop really is. -Raymond]
  94. Pim says:

    [The drop event goes to the frame window. The frame window receives the dropped object from OLE and asks it for its “hdrop” content type (hdrop = “list of file names”). The dropped object is an object in the object-oriented sense (data + code), and it’s the code that’s the scary part.]

    Fair enough. Why is it hard to propagate this “object+code” chunk, as-is, to the unprivileged shell?

    [When you run a program directly, there is no shell. There is nothing to propagate to.]

    [I’ll make a note to cover data object code-behind later this year. Maybe that’ll help.  I think people fail to understand how general-purpose drag/drop really is. -Raymond]

    Please do! I realize I have a large pair of Unix-colored glasses on here that do not account for OLE, which I already suspected to be the source of this difficulty.

    [I find it frustrating that people keep suggesting “fixes” without first understanding what is going on. -Raymond]
  95. Dean Harding says:

    “”Hello world” doesn’t work because you don’t have a message pump.”

    Presumably, the message pumping would be handled by a thread in cmd.exe, not whatever program that is running in the console. The program running in the console would still just write to it’s standard output, and cmd.exe would intercept that and write it to the console window. It would also intercept all the other console-related functions (basically, instead of talking to csrss.exe, those functions would talk to cmd.exe)

    Of course, we’re all just guessing at how it *could* be done, having absolutely no (of very little anyway) idea of how it’s currently done…

    Anyway, that’s a lot of refactoring just to get drag&drop back!!

    [What if there is no cmd.exe? E.g. you Start.Run a console program directly. There is no cmd.exe anywhere. Who runs the message pump? -Raymond]
  96. Gabe says:

    I don’t know anything about programming Macs, but I’ve done my time with X11. There are at least 3 d-n-d standards, however Xt’s is probably the only one supported by terminal emulators.

    As with most things in X11, it is quite simple: once you indicate that you want to accept things dragged to you, you will get an event when an item is dropped on you, and you then process it. When the drag has completed you have the opportunity to find out the type of data and can ask the Xserver for the bits.

    The type of data can be roughly one of: plain text, file(s), or other. There is one global drag buffer, so if somebody does a drag operation before the disk finishes paging in your drop event handler, you get the wrong data. In other words, this DnD facility was designed primarily as a means of transferring filenames from one window to another. No wonder it always seems to work so well.

    Windows has a more generic facility, designed for dragging things like tables, formatted text, charts, and bitmaps. For example, dragging a block of cells from a spreadsheet might show up as a block of cells in another spreadsheet, a mini-spreadsheet in a word processor, a table in an email, tab-delimited text in an editor, a metafile in a vector drawing program, and a bitmap in a paint program. In other words, the mechanism has to be very flexible to be able to deal with formats that might be very complicated. This could mean dynamically loading a library into the drop process to do some of the processing.

    Of course this is all complete overkill for something as simple as copying a filename from one process to another, but transferring filenames is just one of many things the facility was designed for.

  97. Pim says:

    [The process that replaced cmd.exe is the program you wrote. So you’re saying that every console program needs to have a message pump? -Raymond]

    I see what you’re hinting at. So what makes it hard is that the message pump and the privileged display system are in the same layer. Tricky to solve without adding an extra layer equivalent to the Unix ‘getty’ program.

    [I find it frustrating that people keep suggesting "fixes" without first understanding what is going on. -Raymond]

    I wouldn’t be so presumptuous. It just baffled me how this could be a problem. Console handling in windows is pretty counter-intuitive for someone who looked at Unix for too long ;)

  98. Dean Harding says:

    "Why is it hard to propagate this "object+code" chunk, as-is, to the unprivileged shell?"

    Because, as Raymond pointed out above, there may not *BE* a shell (ie. cmd.exe). You wouldn’t expect drag&drop to work only when cmd.exe was involved would you?

    As a quick example, run cmd.exe, then type powershell.exe. That starts up the powershell shell. Next, click Start, then Run and type in powershell.exe. It also starts the powershell shell, but no cmd.exe. You wouldn’t expect drag&drop to work in the first case, but not the second, right?

  99. Pim says:

    [Because, as Raymond pointed out above, there may not *BE* a shell (ie. cmd.exe). You wouldn’t expect drag&drop to work only when cmd.exe was involved would you?]

    No, but I would expect whatever process you can think of to replace cmd.exe as running under the DOS terminal to be equally unprivileged and, thus, equally eligible to receiving propagated events.

    [The process that replaced cmd.exe is the program you wrote. So you’re saying that every console program needs to have a message pump? -Raymond]
  100. Dean Harding says:

    @Pim: But then you would need to add drag&drop support to every console program, instead of just whatever handles the actual console window itself. Not to mention all the console programs that *already* been written.

    It doesn’t have to a "shell" — I only used PowerShell as an example. You could equally run http://ftp.exe or even edit.com.

  101. Gazpacho says:

    "I want to know why the windows console is so many lightyears behind when compared to one of any other OS…"

    Well golly gee, let’s put every other feature of the OS on hold so Microsoft can hack on a feature that terrifies 99.9% of users, to make it more acceptable to people who consider the VT100 the gold standard of user interfaces.

  102. osexpert says:

    [[The process that replaced cmd.exe is the program you wrote. So you’re saying that every console program needs to have a message pump? -Raymond]

    I see what you’re hinting at. So what makes it hard is that the message pump and the privileged display system are in the same layer. Tricky to solve without adding an extra layer equivalent to the Unix ‘getty’ program.]

    I don’t see the problem. Cant the message pump and communication with the "console device" be in the same process as <any console app here>? AllocConsole would create a message pumpthread and handle the works.

  103. Pim says:

    [I don’t see the problem. Cant the message pump and communication with the "console device" be in the same process as <any console app here>? AllocConsole would create a message pumpthread and handle the works.]

    The issue here, if I understand things correctly, is that ‘console apps’, being mostly just DOS apps, have no knowledge of drag-and-drop so the only plausible place to handle it would be one layer up.

  104. Pim says:

    [It doesn’t have to a "shell" — I only used PowerShell as an example. You could equally run http://ftp.exe or even edit.com.]

    Not all of those applications would have a use for dropping file urls. For cmd.exe allowing this would make sense. It would actually even be more useful than Unix, where you get extra surprise features if you drop a file url on a window running, say, pine :)

  105. Pim says:

    [Well golly gee, let’s put every other feature of the OS on hold so Microsoft can hack on a feature that terrifies 99.9% of users, to make it more acceptable to people who consider the VT100 the gold standard of user interfaces.]

    I think that in the business world there are droves of people that would have been really thankful if Windows had spawned a command line shell that was less of a third class customer in the larger ecosphere. But that’s not really a technical discussion and the only statistics I have on this would require a doctor with a flashlight to pull them out.

  106. steveg says:

    [I can’t believe anybody used anything other than Alt+D to get to the address bar. That’s why the D is underlined in "Address"… -Raymond]

    I’ve always used F6 (that’s a Win3.0 habit I’ve kept). Never even noticed the underlined D (not that IE7 has the word A&ddress anymore).

  107. Duncan Smart says:

    Try this – it’s a great wrapper for any console-based shell such as cmd.exe or PowerShell and supports drag & drop: http://sourceforge.net/projects/console

  108. anonymous says:

    So that’s the reason why. I was wondering. Btw, almost the entire article http://en.wikipedia.org/wiki/Features_removed_from_Windows_Vista was written by me. Can you explain all the removals gradually over time in your blog posts?

    [I know you may find it hard to believe, but I am not consulted on every Vista feature decision. (I wasn’t consulted on the theming one either.) I in fact know nothing about any of the things listed in that page. You’ll have to find somebody else to do your Wikipedia research for you. -Raymond]
  109. anomymous says:

    Once again: Has this been fixed on Windows XP? Or does Microsoft intentionally leave a known security hole open to push its lousy Vista?

    [There is no known security hole here. The change in Vista was a “defense in depth” measure, not a defense against an actual vulnerability. Belt and suspenders. Seat belt and air bag. Life jacket and floating seat cushion. -Raymond]
  110. Dean Harding says:

    That "Features removed from Windows Vista" (wouldn’t "Features removed IN Windows Vista" be a better name? The current name implies that those features had been intended for Vista, but were later removed) demonstrates just how important backwards-compatibility is. Remove a couple of minor features, and someone writes up a whole Wikipedia article about it!

    I mean, who seriously considers the fact that there were multiple status icons for each network adapter a "feature" of XP? I spent more time trying to turn the damn things off than I did actually looking at them…

  111. Gazpacho says:

    You’d be amazed the kind of things Wikipedia editors care about.

  112. jd2066 says:

    @Dean Harding: I don’t think all of the things on that list are minor. For instance the Sound Recorder is now missing the ability to open and edit audio files. Which sucks if you want to record something as a test to see if an input is working and then play what was recorded.

  113. jd2066 says:

    Hmm. Actually thinking about it. That might be a minor thing too.

    Anyways it does make since for the article to be there as it’s about Windows Vista and it’s feature history. If it was shorter it would probably be on the main Windows Vista wikipedia page.

  114. Miral says:

    As I said before, I think a better “defense in depth” strategy would have been to get all the console-handling code out of the system-level process to begin with.  Then this sort of thing (and the crash bug I mentioned that takes out the whole system) wouldn’t have been a problem.

    Ideally, when an app wants to create a new console, the system could just inject a common DLL into the process and get it to run a thread to handle the rendering and other Windows messages at the same security level as the process itself.

    One complication of this idea is that I think it’s possible for a console to survive past the lifetime of the creating app (by passing handles between processes), so “ownership” of the console would have to be passed on to another process when the owning process exits if that’s the case.

    So maybe having the dedicated “console.exe” is a better idea after all.  Have one console process per security level (ie. one for low-integrity, one for normal-integrity non-elevated, another for high-integrity elevated, etc) that does all the work, rather than keeping it in CSRSS.  It should be simple enough to make the existing code in CSRSS pass requests down to the console-handling processes, or spawn a new one if needed.

    [A redesign of the console system is a rather drastic move for a defense in depth measure, don’t you think? Who knows what security vulnerabilities the rewrite may inadvertently introduce. Change has risk. Big change has big risk. -Raymond]
  115. jd2066 says:

    @Dean Harding: I don’t know, I haven’t used that program. I just mainly used Sound Recorder as a quick test to see if audio inputs were working before using other programs with it. Since I could just use a different program to test audio inputs it’s not really a big deal. I was just used to doing it with Sound Recorder.

    Back on topic I used Drag and Drop to Command Prompt quite a bit with XP and I still try it in Vista since I’m used to it and forget it’s not there anymore.

    One reason it’s better then the copy path to clipboard item is because then you can you can keep something else in your clipboard.

    There were times when I wanted to set a new path but not override my clipboard so DND was quite useful there.

  116. Dean Harding says:

    jd2066: What’s wrong with Audacity?

    Of course, the issue that affects you personally is never "minor" but you’ve got to wonder how many people actually ever even *used* sound recorder (I tried it once or twice back in Windows 95 and then never touched it again).

    I think we’re drifting a bit too far off topic now anyway :-)

  117. Matthieu says:

    I don’t get it. People want to use GUI concepts on a COMMAND PROMPT? A Commend Prompt is NOT a GUI.

  118. anonymous says:

    “There is no known security hole here.”

    According to your description, there is. Csrss seems to be vulnerable to a shatter attack, since it allows to recieve OLE objects and automatically executes embedded in there.

    Now if this is not a security hole, then either Csrss doesn’t execute code in OLE objects it receives, or it doesn’t accept any OLE objects received, or isn’t able to receive OLE objects at all. Which one is it?

    [Note the word “known” in “no known security hole”. This is “defense in depth”, where you add an additional layer of protection in case there is an unknown security hole. CSRSS accepted OLE objects but did not execute code from them. There is no actual vulnerability, but there is potential for a vulnerability (if a bug in CSRSS, OLE, or some other layer causes the code to execute by mistake). I believe I’ve already noted that I will take up the topic of “defense in depth” in a later entry, since it seems people don’t quite get it. -Raymond]
  119. jd2066 says:

    @Matthieu: But the Command Prompt is a GUI interface to a command line so it does make since to apply some GUI concepts to it.

Comments are closed.