Riffing on Raymond – the purpose of an operating system


 

In Raymond’s post today (Adding flags to APIs to work around driver bugs doesn’t scale), Raymond wrote:

Perhaps it’s just me, but I don’t believe that workarounds for driver issues should become contractual. I would think that one of the goals of an operating system would be to smooth out these bumps and present a uniform programming model to applications. Applications have enough trouble dealing with their own bugs; you don’t want them to have to deal with driver bugs, too.

My personal take on this is that the entire PURPOSE of an operating system is to smooth out the bumps and present a uniform programming model to applications.  Applications don’t need to know about the weird behavior of of the u765 floppy disk controller that only shows up when there are more than two floppy drives (I got a free 72 hour trip to France because of that particular issue).  They shouldn’t need to know that a certain models of network adapter will sometimes hand the OS a packet that is half filled with garbage. Applications shouldn’t have to know the low-level commands needed to drive a printer.  

IMHO, the ONLY reason for an operating system to exist is to hide the details of the hardware from the application.  Everything else is sugar.

People forget what the world was like when they would buy a word processor and it would come with a list of the two dozen or so printers that were supported by the application.  If you wanted to use the word processor but didn’t own one of the listed printers, you were out-of-luck.

For those of you that are now going to say “Aha!  Larry now thinks an OS shouldn’t contain a web browser”, that’s not at all the case.  An HTML renderer is a very high level special case of hiding the details of the hardware from the application.  In the case of an HTML renderer, it’s job is to provide a uniform interface that can be used to render HTML content to the screen, just like a printer driver provides a  uniform interface that can be used to render content to a printer.  Similarly an OS should contain a TCP/IP stack and network programming layer that allows for application authors to avoid understanding the icky details of interoperability (it’s HARD to write a high quality TCP/IP stack, and application authors shouldn’t have to).

 

But I still stand by my original assertion: The only reason for having an operating system is to hide the details of the hardware from the applications that run on the computer.

 

Btw: This is MY opinion.  I want to repeat that.  MY OPINION.  Others undoubtedly feel otherwise.

 

Comments (32)

  1. Anonymous says:

    Larry,

    Your opinion is well respected from my end.  I for one and thoroughly enjoying the responses to Mr. Chen’s posts as it shows how totally impractical most programmers are.  Almost everyone has the opinion of "make the other guys fix it" but that just doesn’t fly.

    And because people don’t try to understand that mentality that is Microsoft, they will never understand why Microsoft is NUMBER ONE.

    James

  2. Anonymous says:

    The HTML Renderer I think is a bit of a stretch.  I mean, can’t that also apply to having Windows Media Player as part of the OS (or, say, bundled with the OS) … let’s try it:

    "In the case of an Media Renderer, it’s job is to provide a uniform interface that can be used to render Movie content to the screen, just like a printer driver provides a  uniform interface that can be used to render content to a printer."

    Sweet, just plug anything in there and you can say that any content that isn’t instantly human readable or human useable needs an OS component to provide a rendering, which leads to fun things.

    I know this is being simplistic, but I think I need an easy way to print to the printer more than I need IE tangled around the OS.

  3. Peter Ritchie says:

    Where do you draw the line though?  To compensate for drivers that may be particularly buggy you could spend several milliseconds determining that the bug exists (or does not exist) before deciding on an acceptable plan of execution.  Why bog down everyone’s experience because of a hand full of buggy drivers?

    Raymond’s post was in response to a response to a post about dealing with network-attached storage devices with "buggy" drivers, so it’s not like exact code can be loaded upon OS boot.  In the referenced case the "bug" could not be detected until after the first response to a file list request.  In this case a plan of execution would have to be devised upon first access to the device, and probably all subesequent accesses because the OS is not informed of driver updates for network-attached devices.  But, you certainly don’t want to request the first batch of 100 files just to try and detect if that particular device was on the network ruinly everyone’s experience–expecially considering the bug appears in "fast mode".

    Yes, it’s noble and even suggested to hide all possible device implementation details from applications and users; but, it may conflict with other requirements in some cases.

  4. MGrier says:

    A good counter-example was the past decade where very rapid innovation was going on in the video space.  DirectX really had no choice but to try to expose the capabilities of each device.  The hardware ten years ago was very differentiated in even basic capabilities and the GDI model which does hide most of the differences between video cards could not deliver the performance required.

    (I’m not going to mention OpenGL more than in this one parenthetical paragraph.  Maybe it could have been the API of choice but still at the time you had to use custom extensions for each device to get optimal performance.  It was not clear that anyone could either drive eventually towards unity as clearly and effectively as Microsoft eventually did in the most recent DirectX releases.)

    This is a case where hiding the hardware would have killed things.  The last 10 years have been bake time for video cards and now the basic capabilities are well understood enough that future plans lean more towards a standard set of required capabilities.  I have no details other than the obvious standardization of pixel shaders which had the opportunity to go exactly the opposite way and start a pixel shader ISA war.

  5. Peter, that’s a large part of the job of an OS – to compensate for drivers that are particularly buggy.

    For Windows, there’s alway the baddriver solution – Somewhere in Windows is a list of "known bad" drivers which Windows will refuse to load (I don’t know how it works or anything beyond the fact that I’ve seen it in action).

    xix: Do you agree that it’s a good thing that applications can render multimedia content without having to build a custom decoder for that content (or license one from a 3rd party)?  If you believe that 3rd party apps like winamp and iTunes should be able to use platform multimedia services without having to write their own, then yes, it does make sense to include a media renderer in the OS.

  6. Anonymous says:

    I went to a college that specifically had an CS: Operating Systems degree track and that’s exactly what they told me.  Everything I’ve seen in the 15 years since has reinforced that belief.  "Operating Systems exist to abstract hardware from applications" is not just an opinion to me, it’s a fact.  Something we lose sight of too often, trying to make it an "experience".

  7. Anonymous says:

    I like the article but couldn’t help but smile at:

    "it’s HARD to write a high quality TCP/IP stack…"

    Microsoft, of course, gave up on their own TCP/IP stack and switched to using BSD’s high quality one.

  8. Stuart, we did?  For NT 3.1, we used the BSD stack, but for all subsequent releases, we’ve used one written in-house.

  9. Anonymous says:

    Don’t forget that Windows has been shipping with a hypertext viewer since 1990 (WinHelp). In the late 90’s, though, they switched the format to HTML, in part because it’s much easier to author than the old format. So now as along as the OS has to include an HTML rendering engine, why not use it for things like customized folder views or control panels? Of course, it makes sense to create a shell around it (IE) to for people to browse the web.

    Come to think of it, if Windows didn’t include a web browser, how would your average user acquire one?

  10. Anonymous says:

    >> xix: Do you agree that it’s a good thing that applications can render multimedia content without having to build a custom decoder for that content (or license one from a 3rd party)?

    Well, I disagree. It should be in a separate SDK/library, available free or pay, offered by you or any other party.

    It is not a part of the operative system.

    And, even if you think it should, it could be there WITHOUT the player application. You know, ShWvDoc.dll and IEXPLORE.EXE are not the same thing. If MS just offered the renderer embedded but no integrated browser (or even, a bundled but uninstallable browser) nothing would have happened.

  11. Anonymous says:

    I’m currently a CS student at PSU, and the definition that we’ve been pounded with is that the purpose of an operating system is resource management and arbitration. In other words, the purpose of the operating system is to provide a framework for sharing resources such as CPU cycles, memory, disk space, and I/O devices. In my personal opinion, the modern operating system has two purposes: Provide resource management and provide a rich environment for developing applications. Pretty much everything in the kernel (and associated drivers) satisfies the first part of that definition. The HTML renderer, for instance, falls into the second part of that definition.

    Any chance of hearing more about the u765? That sounds like an interesting story.

  12. Mike Dimmick says:

    Traditionally an OS really was just something that abstracted the applications from the hardware. It’s now become something that is a union of a) the traditional OS, b) a really wide selection of application development libraries offering higher level abstractions over a), and c) a small number of applications that are intended to be compelling enough to attract users. All operating systems – call them operating environments if you like – are now like this, and the increase in category c) is not slowing down. In this arena Windows is probably the least-featured of the lot (at least for desktop OSs).

    The HTML renderer falls firmly into category b) and has done ever since IE 3.0 I think. The ‘Internet Explorer’ you see is almost entirely category b) – published APIs – with a very thin UI wrapper falling into category c). Windows Media Player (c) is almost entirely wrapped around DirectShow (b). Up to you whether you count the Media Player ActiveX control as being b) – I think it is.

    You _could_ have an OS that would be only category a) but I don’t think you’d find too many takers.

  13. Anonymous says:

    Larry, how do you feel about the case where an application may need access to the "nitty gritty details" that an OS exists to hide?

    In you example of an IP stack…

    I write network applications that at times need access to information that I can get using socket options on other OSs.  For example, in Linux I have an application that can get the destination address from an incoming UDP packet.  Set the DSTADDR_SOCKOPT socket option and then examine the cmsghdr structre of an incoming packet using MSG_PEEK.  It’s actually quite handy.  

    There is no such thing, AFAIK, in Windows.  If there is, please, someone feel free to enlighten me.  It’s actually worse in .NET 1.x.  There seems to be an assumption in that framework that nobody uses UDP. Sigh.  

    I’m really not flaming Windows here in comparison to Linux.  Believe me, I had to dig the Richards Unix Net Programming Bible to find out how to do this.  I use Windows day to day, BTW.  But, there’s one example of the OS "smoothing the bumps" that actually hurts some app developers.  I say, at least give me the choice of getting at the details when I need to and don’t assume I’ll never use it.  Granted, this falls into the 95/5 rule (95% don’t need it).  But, it’s makes like hard for the 5% who do.  

    BTW, Stuart, if the Windows stack was the BSD stack I could do this in Windows.  They’re similar from an API perspective, but Windows is not a copy of BSD.

    – Coleman

  14. Anonymous says:

    > For Windows, there’s alway the baddriver solution –

    > Somewhere in Windows is a list of "known bad" drivers which

    > Windows will refuse to load

    If only.

    Some drivers that are built into Windows need to be included in that list.  Why aren’t they?  Why isn’t the baddriver solution even allowed to be the solution in these cases?

  15. Slackware comes to mind.

  16. Anonymous says:

    "IMHO, the ONLY reason for an operating system to exist is to hide the details of the hardware from the application."

    So what bit of hardware does an HTML renderer hide from applications?

    Oh, it doesn’t. It just hides lower-level libraries, (which hide lower-level libraries, …) which talk to the kernel (or to a video driver with an interface defined by the kernel/core OS lib), which hides the details of the hardware.

    And how do other applications talk to the "web browser"? Oh – they don’t? No, they talk to the HTML renderer. Nice of you to switch between "HTML renderer" and "Web browser" as if the two were interchangable when one is a useful library that can be used by other applications to render HTML content, while the other is an end-user application.

    "We decided to include a useful shared rendering library for everyone to use." is a _completely_ different statement than "We decided to include a web browser, place it on the desktop, and raise the costs (directly or indirectly) for OEMs who ship Navigator at least as visibly as IE so they won’t do it."

    (See http://www.groklaw.net/article.php?story=20060404153340949 in particular quoted paras 158-159, 167, 171-172)

    You’re trying to justify the second of those statements by saying it’s really the same as the first, which is *really* insulting.

    Stick to the technical stuff, you’re much better at it.

  17. Anonymous says:

    Larry wrote:

    > But I still stand by my original assertion: The only reason for

    > having an operating system is to hide the details of the

    > hardware from the applications that run on the computer.

    But surely that statement is meaningless without defining the level of abstraction that you think an OS should provide? Otherwise any piece of software from a kernel-mode device driver that reads/writes hardware registers all the way up to a user-mode HTML renderer (or PDF renderer or word processor or whatever) can be defined as a hardware abstraction and should therefore belong in an OS.

    Mike wrote:

    > You _could_ have an OS that would be only category a) but I

    > don’t think you’d find too many takers.

    Sure you would, just not from desktop PC users. The realtime/embedded world has no need for HTML renders and whatnot so the operating systems that sell well into that market, like WindRiver’s VxWorks (which is widely used in the telecoms business and also runs on the Mars rovers), would fit your definition of a ‘traditional’ operating system I think.

  18. Anonymous says:

    Wednesday, April 05, 2006 1:47 PM by Stuart Rowan

    > BSD’s high quality one.

    BSD’s high quality one was accomplished after fixing their low quality one.  Once upon a time the manual pages even mentioned which game it was that exposed bugs in BSD’s original implementation of sockets.

    Up to a point it the occurence of bugs isn’t the real problem, the real problem is whether the bug maker will provide customers with fixes for serious bugs.

  19. >> Microsoft, of course, gave up on their own TCP/IP stack and switched to using BSD’s high quality one.

    Care to see a video?

    http://channel9.msdn.com/ShowPost.aspx?PostID=116349#116349

  20. Anonymous says:

    The ie/wmp problem is that of modularity…microsoft is free to provide anything with the os, but the user should be able to not install/remove them without leaving any trace and install his own modules to provide those services…so that for example all the html-based help (vstudio comes to mind) would use firefox/opera and multimedia decoding would be provided by winamp/vlc…etc

    Of course that would require open apis and collaboration with the community, and the day microsoft starts to pursue such goals will probably have us sporting long white beards 🙂

  21. Anonymous says:

    Something changed in the stress mix last week, and I’ve been swamped with stress failures, that’s what…

  22. Mike Dimmick says:

    Coleman: I’m struggling to understand what it is you need that the recvfrom() function does not give you, unless you want a way to find out the address _before_ you handle the packet. Even then I think you can use MSG_PEEK with recvfrom().

    Andrew: that’s why Microsoft has Windows CE. If you really do need to minimise the size of your device’s firmware image, but still want to employ Windows development expertise, it can be a good choice. I don’t know if you can configure Windows XP Embedded so that it doesn’t include the HTML renderer, although I suspect you can. But this is a highly specialised market, for situations, in general, where you know the complete set of applications which will run on the platform, and can therefore select exactly which development libraries and supplied applications you’re going to include.

    I mostly work on devices that run either Windows Mobile – which is a Microsoft-specified set of OS components, development libraries and applications married to a specific OEM’s drivers and any extension libraries and applications they care to add – or an OEM’s selection of components from the Windows CE box, again plus their own additions. In both cases you live with what Microsoft and/or the OEM supplied.

  23. Anonymous says:

    Mike, I need to know the address that the packet is destined for.  Where does recvfrom give me that information, even using MSG_PEEK?  Yes, I need it before I process the packet (or at least, readily available as I process the packet).   Is that information part of the raw buffer or something?

    Another example, specific to Windows CE.  How does one reliably obtain the subnet mask?  I’ve used the trick where you look in the registry (HKLMCommadaptername, etc.), but that’s simply not always reliable.  The information doesn’t show up there when it’s changed, for example.  The registry path is different at times, relying on HKLMCommPCIadaptername.  Huh?

  24. PatriotB says:

    "The ie/wmp problem is that of modularity…microsoft is free to provide anything with the os, but the user should be able to not install/remove them without leaving any trace and install his own modules to provide those services…so that for example all the html-based help (vstudio comes to mind) would use firefox/opera and multimedia decoding would be provided by winamp/vlc…etc"

    Oh man.  Can you really imagine if this were the case?  Think of the test matrices!  Not only do you have to test your apps against all the various Windows/IE combinations, you also need to test against Firefox versions, Opera versions, etc.  And the *slightest* difference between them could hose your application.

    If every component of Windows were designed such that it could be swapped out by a third-party implentation, we’d be seeing Vista in about 2050 or so.  Development and support costs would go through the roof, not just for MS but for all ISVs.  During the remedy phase of the MS-DOJ trial, MS explains this in more detail.

  25. Anonymous says:

    I’m aware of that explanation, but I still think that with a clear api and maybe a process to certify that the modules render the content as intended, there wouldn’t be that many problems…after all, it would be the user’s responsibility to install the alternative modules and a lot of users don’t want to deal with that or won’t be allowed. And as for thesting, it would be actually easier both for microsoft and the isvs. Modularity _is_ good…

  26. Anonymous says:

    "The ie/wmp problem is that of modularity…microsoft is free to provide anything with the os, but the user should be able to not install/remove them without leaving any trace and install his own modules to provide those services…so that for example all the html-based help (vstudio comes to mind) would use firefox/opera and multimedia decoding would be provided by winamp/vlc…etc"

    And then Windows users and programmers would find themselves in the GNU/Linux nightmare world of non-standardisation. (Is that a word?)

    No thanks…

  27. Anonymous says:

    @coleman:

    IP_HDRINCL with raw sockets is what you need. I use it all the time (for custom IPSec and related implementations)

  28. Anonymous says:

    Ok, I did some googling and digging in the MSDN and to retrieve the subnet mask on WinCE 5.0 you should use the GetAdaptersInfo IP helper function.  That’s great for that version of CE, but what about older versions?  Am I stuck with the registry hack?

  29. Anonymous says:

    Here’s GetAdaptersInfo for CE 4.2:

    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcetcpip/html/_wcesdk_win32_GetAdaptersInfo.asp?frame=true

    The page says the minimum CE version is 3.0.

    I used it in computing a guess of a subnet mask because I couldn’t find any method that is documented to be reliable.