When you right-click on the desktop or in an Explorer window and select "Sort by... Name", why doesn't it actually sort by name?
Because "Sort according to the default sort order for the column whose title is... Name" is too long.
The default sort order for the first column (whose name is "Name") is to sort by name, mostly. The actual sort order for the desktop and file folders (note: implementation detail, subject to change at any time) is really a two-tiered sort. First, the items are sorted into buckets; the first bucket contains the virtual folders, the second bucket contains file folders (directories), and the third bucket contains regular files. Sorting within the first bucket is done by the "item order"; sorting within the second and third buckets is by name. (And it's not by ASCII code any more. There's also special treatment for numbers and periods.)
The "item order" (a term I just made up) for My Computer, My Documents, and Recycle Bin were specifically chosen in Windows 95 so that the icons on the desktop appeared in that order. I believe it was in Windows 2000 that the order was changed to My Documents, My Computer, Recycle Bin in order to put the focus on document management.
(And those who have read Appendix A of my book will know what sort of application compatibility problems arose from that simple change.)