Just change that 15 to a 1


It would be nice and easy to just change that 15 to a 1.

If only it were that simple.

In the case described in that article, it's not that a single operation was attempted fifteen times in a loop. Rather, the fifteen operations were scattered all over the program. Suppose, for example, that the network operation was "Get the attributes of this file." The program might be filling in a file listing with several columns, one for the icon, another for the file name, another for the file author, and the last one for the last-modified time.

for each filename in directory {
 list.Add(new ListElement(filename));
}

Well, that doesn't access the same file fifteen times. Oh wait, there's more. What happens when it comes time to draw that list element?

ListElement::DrawIcon()
{
 if (m_whichIcon == don't know)
 {
  m_whichIcon = GetIcon(m_filename);
 }
 draw the icon for the element
}

// with this common helper function
GetIcon(filename)
{
 if (filename is a directory) {
  return FolderIcon;
 } else {
  return PieceOfPaper;
 }
}

Okay, getting the icon accesses the file once. You can imagine a similar exercise for getting the file's last-modified time. What else?

ListElement::GetAuthor()
{
 if (m_author == don't know) {
  AuthorProvider = LoadAuthorProvider();
  m_author = AuthorProvider->GetFileAuthor(m_filename);
 }
 return m_author;
}

// where the author provider is implemented in a
// separate component
GetFileAuthor(filename)
{
 if (filename is offline) {
  return "";
 } else {
  ... open the file and get the author ...
 }
}

Getting the author accesses the file once to see if it is offline, then again to get the actual author (if the file is online).

So in this simple sketch, we accessed the file a total of five times. It's not like there's a 5 in this program you can change to a 1. Rather, it's a bunch of 1's spread all over the place. (And one of the 1's is in a separate component, the hypothetical Author Provider.)

It reminds of a story I may have read in John Gall's Systemantics: There was a server product that was having problems under load. Once there were more than thirty simultaneous users, the system slowed to a crawl, but the customer needed to support fifty users. At a meeting convened to discuss this problem, an engineer joked, "Well, we just have to search the source code for the #define that says thirty and change it to fifty."

All the people at the meeting laughed, except one, who earnestly asked, "Yeah, so why don't we do that?"

Comments (51)
  1. Anonymous says:

    Oh, the memories.

    I’ve had to fix code like this before.  The sad part is, the same developer made the exact same mistakes on the next version of the program.  When will people learn that just because it is easy to code something, it doesn’t mean that executing that code won’t be expensive.  

    I doubt the developer in the article even worried about performance.  Preoptimization might be bad, but uniformly slow software due to no consideration for performance has cost the company I work for a large amount of money.

  2. Anonymous says:

    I recently had a problem where any ShellExecute wouldn’t return for over a half-hour. Since the taskbar clock didn’t change, I got used to seeing the one particular time, so when it eventually changed I immediately noticed and computed the difference.

    The difference was 34 minutes. I could imagine a timeout of 30 minutes somewhere, or maybe even 32 (power of 2). But 34? Then I realized that 2048 is just 8 seconds more than 34 minutes.

    Of course nobody (hopefully) is stupid enough to put in a 2048 second timeout somewhere. Unfortunately it’s quite likely that there’s a 16 second timeout, which is retried 7 more times, and when that function fails is retried 3 more times, and when that function fails is retried 3 more times.

    Eventually I found the one app that wasn’t responding to messages and was causing the whole system to not work. Killing that app made ShellExecutes complete immediately and allowed the system to go to sleep.

  3. Anonymous says:

    That great story you’re remembering about going into the code and increasing the "thrashing limit" comes from none other than Douglas Hofstadter.  I think it was in one of this Scientific American columns in the early ’80s.  It’s a good example of what evolutionary biology calls a "spandrel," an accidentally emergent phenomenon.

  4. Anonymous says:

    I wouldn’t comment elsewhere but I know you’re a language geek and I’m a pedant and also reading blogs to avoid doing any homework, so:

    "Rather, it’s a bunch of 1’s spread all over the place."

    That apostrophe… burns…

  5. Anonymous says:

    It would be nice if the programmer could write the program in a logical and the OS should help him/her out.  If a file is accessed a few times it should be cached by the OS so it should be the same thing as the programmer only accessing it once.  Maybe this isn’t the way with Windows.

  6. Anonymous says:

    —-"Rather, it’s a bunch of 1’s spread all over the place."

    That apostrophe… burns…"—

    Actually, it’s more a question of changing fashions. Forty years ago it would have been quite common but now the only time the apostrophe is recemmended for use in plurals is after individual letters, and possibly words such as "if’s and but’s". The use of the apostrophe after numerals — 1’s and 2’s –,   decades — the 1990’s, and initialism — TV’s, CD’s– is now deprecated.

  7. Anonymous says:

    I’d argue it may have been fashionable, but simply grammatically incorrect :P

    ‘1’s status as a number is assured’ is fine, as is ‘1’s a most excellent digit’ – either as possessive or substitutive. To use it in plurals is just… a mistake?

    See – http://en.wikipedia.org/wiki/Grocer%27s_apostrophe#Greengrocers.27_apostrophes

    [Call me old-fashioned, but I grew up with such quaint rules as “spell out numbers less than 100” and “to form the plural of numbers, add an apostrophe and s” (which is I believe still the recommendation of the MLA though I haven’t checked recently.) -Raymond]
  8. Anonymous says:

    That’s pity that none of programming languages has paradigm like:

    time=GetExecTimeEsctimate(func(param));

    if(time < reasonable)

       func(param);

    or better it whould be

    time=func(param,GET_EXEC_TIME);

  9. Anonymous says:

    Nekto2, um, er, can you say halting problem?

    In other words, how do you implement GetExecTimeEstimate without actually executing func?

  10. Anonymous says:

    Andrew: it’s still a pity we can’t do that though.

  11. Anonymous says:

    That #define is amusing. At work we have a case where the solution is to dramatically decrease the define to improve performance. Too much mutex contention because of overly broad protected data areas. Finer-grained protected areas look likely to fix the problem. The joys you a can have with code that makes safe assumptions for Pentium IIIs and RAM maximums from that era when it’s run on 4 or 8 modern cores with tens of gigabytes to protect.

  12. Anonymous says:

    Mike’s right – the first call to the file should load up its metadata, and the next one (within reasonable timeouts) should just hit the cache. I admit that I’m inured to windows’ weirdness and initially considered writing a File object cache in my app. At least I can be sure that will behave consistently.

  13. Anonymous says:

    It’s not grammatically incorrect, it’s orthographically incorrect.  We’re talking about a writing convention here, which has nothing to do with the grammar of English (which doesn’t concern itself with punctuation).

  14. Anonymous says:

    Even if the local computer utilized the cache, what if the file was on the network.  You would still have to spend the time to talk over the network just to get the other values from the networked computer’s cache.

  15. Anonymous says:

    — That apostrophe… burns… —

    If that apostrophe burns, your skin must be melting because you ended a sentence with an ellipsis but without a period. Check it out:

    http://en.wikipedia.org/wiki/Ellipsis#Ellipsis_in_English

    If you don’t believe me, read any of the opening crawls of the Star Wars movies….

  16. Anonymous says:

    The Hofstadter story is also from Chapter 10 of GEB.

  17. Anonymous says:

    djimi, I think it’s impolite to criticise without leaving at least one appropriate mistake of your own. Seems like a writing convention of the Internet anyway! ;-) Thanks.

  18. Anonymous says:

    Andrew Feldstein: there are some cases when you do. OS knows is the files resides on HDD, network or mem-fs. Also is has info about last access to whole network resource or even if network cable was plugged-out – you could ask of estimate access to any file on that resource and that would be the same. And the language and libs just made specially that way to hide this information from you.

    Imho http://joelonsoftware.com/articles/LeakyAbstractions.html describes it.

    There should be some access to that info – you can’t time-optimize timeouts when you have no info to base optimization decisions on.

    ps. Ok that function could return – "Estimate time is uknown" ;)

    pps. Actually it should be:

    GetTimeEstimate(…);

    TestTimeEstimate(…);

    And try to implemet those so GetTimeEstimate(TestTimeEstimate(…)); will always return a value ;)

  19. Anonymous says:

    At a meeting convened to discuss this

    > problem, an engineer joked, "Well, we just

    > have to search the source code for the

    > #define that says thirty and change it to

    > fifty."

    >

    > All the people at the meeting laughed,

    > except one, who earnestly asked, "Yeah, so

    > why don’t we do that?"

    What kind of meeting was this?  With just one manager in attendance?

     

    Tuesday, September 05, 2006 4:37 PM by Andrew Feldstein

    > Nekto2, um, er, can you say halting problem?

    > In other words, how do you implement

    > GetExecTimeEstimate without actually

    > executing func?

    Mr. Feldstein, if you can say halting problem, then you know that the implementation or non-implementation of GetExecTimeEstimate does have to be without actually executing func.  If you perform a limited analysis only considering hypothetical cases which execute func, then you don’t prove that GetExecTimeEstimate reduces to the halting problem.

  20. Anonymous says:

    I just found out the Winzip’s shell Context menu extension does just that – it calls GetFileAttributes() several times, presumably once for each menu item. Over a slow share, it takes several seconds, during which Explorer’s window is frozen.

  21. Anonymous says:

    Mike, please RTFA. This is a timeout on a network file we’re talking about – on a distributed network that tried a bit too hard to hide transient failures. The operation never succeeds, the information never gets a chance to be cached. The operation fails N times, each because of a timeout (if the connection was dropped, every attempt tries to reconnect and has to time out)

  22. Centaur says:

    In such a situation, the application should have a cache of everything, and a reasonable expiration policy. For example, in an Explorer-like application, the cache would expire when the user hits F5.

    The application is also responsible for not doing slow operations unless explicitly asked. It is common knowledge that it is a bad idea to access shell namespace items with SFGAO_ISSLOW attribute; similarly, Explorer and other file management applications should not attempt to extract icons from .exe and other such files when the files are accessed over the network. (Especially because there might be a vulnerability in the icon extraction code, in which case it becomes remotely exploitable.) I’m not even talking of image and video preview over the network.

    [“A cache of everything”? How much of “everything” is “everything”? Remember that Explorer is a combination of multiple components, many of which aren’t written by Microsoft (e.g. that WinZip context menu extension mentioned above). If a context menu extension reads the file, should it also talk to the “everything cache” and say “Hey, please cache these 500 bytes I read from the file”? Building a thumbnail reads the entire file. Should the entire file go into the “everything cache”? Note also that blocking icon extraction over the network will result in some corporate networks never extracting icons! (They keep everything on the network.) A Start menu with no icons looks kind of, well, lame. -Raymond]
  23. Anonymous says:

    —-"djimi, I think it’s impolite to criticise without leaving at least one appropriate mistake of your own. Seems like a writing convention of the Internet anyway! ;-"—-

    The principle is known as Hartman’s Law of Prescriptive Retaliation which states that "any article or statement about correct grammar, punctuation, or spelling is bound to contain at least one eror".

    The point you fail to understand about the apostrophe being used in the three cases where it is now deprecated, is that it was the orthographic convention some forty or fifty years ago and change has come about gradually

  24. Anonymous says:

    —- "The point you fail to understand about the apostrophe being used in the three cases where it is now deprecated, is that it was the orthographic convention some forty or fifty years ago and change has come about gradually" —-

    Which I’ve now discovered thanks to this discussion I hijacked slightly :-) (Sorry about that btw.) It’s interesting because I can certainly see the use for making the plurals of lowercase letters, although I’d be tempted to write ‘a’s, however silly that might look. I can’t not cringe at the sign outside the nearest Internet café boasting of 50 PC’s though.

  25. Anonymous says:

    Norman Diamond, Nekto2: Estimating the runtime of a function based on analysis of its code is possible, but brutally difficult. Doing it based on already compiled code is even harder. Besides OS scheduling problems, the estimator may have to figure out the complexity of the operations the function performs and how those relate to the input parameters.

    [Consider this function:

    void f()
    {
     if (GetTimeEstimate(f()) != INFINITE) Sleep(INFINITE);
    }

    What is the value of GetTimeEstimate(f())? How about this function:

    void g()
    {
     Sleep(getch() * 1000);
    }

    What is GetTimeEstimate(g())? -Raymond]

    For example: Computing the Fibonacci sequence recursively vs. iteratively vs. recursively with memoization. One is exponential and the other two are linear in complexity.

    What I would try for instead would be to use the thread context to set a timeout and then have a function that can test if a timeout has been exceeded:

    Calling code:

    // The thread has 3 seconds to reach the

    // ClearThreadTimeout line from here.

    SetThreadTimeout (3000);

    LengthyOp ();

    ClearThreadTimeout ();

    Called code:

    void LengthyOp ()

    {

       while (!ThreadTimeoutExceeded()) {

           DoAQuickOperation ();

       }

    }

    The above does not account for functions that ignore the timeout, nor does it account for exceptions thrown, OS scheduling and lots of other things. But it would provide a way to delineate a region of code and allow the passing of a “deadline” up the call stack without making the deadline an explicit parameter. And I think that while this isn’t a total and absolute solution, it would allow for some rudimentary real-time control.

  26. Anonymous says:

    I grew up with such quaint rules as “spell out numbers less than 100”

    Like in “change that 15 to a 1”, you mean?

    [That was intentional, because the hypothetical line of code doesn’t say #define LIMIT fifteen. Would you be happier if I had written “Change that ’15’ to a ‘1’”? -Raymond]
  27. Centaur says:

    2Raymond

    > If a context menu extension reads the file, should it
    > also talk to the “everything cache” and say “Hey,
    > please cache these 500 bytes I read from the file”?

    If getting file attributes is so expensive that we are talking of a cache, reading the file is out of the question.

    > Building a thumbnail reads the entire file. Should
    > the entire file go into the “everything cache”?

    First of all, if reading the file is expensive, we shouldn’t be doing it without user’s request. Second, if the user requests and we get to read the file, yes, we should hold onto it for as long as plausible, at least until the user leaves the directory. Because when (s)he sees the thumbnail, (s)he may want the entire image, and it would be rude to lose it and have to redownload it. (It would also be rude to download all the 1000 4-megapixel photos in the directory and fill up the whole memory and %temp%.)

    > Note also that blocking icon extraction over the
    > network will result in some corporate networks
    > never extracting icons! (They keep everything on
    > the network.) A Start menu with no icons looks
    > kind of, well, lame.

    Then it should be a matter of policy. A registry setting of a list of file path prefixes on which to disable implicit file access would be just fine. (Extracting icons from a floppy isn’t very exciting, either.) If the domain administrator considers the local network capable of handling the extra load, he does not include ‘\’ in this list by default.

    [You’re the one who proposed a cache, not me. Are you saying that if the user asks for a thumbnail of a 2MB image on the network, we should cache the entire file in memory? Yet you also say it’s rude to fill memory with these cached files. I’m confused. You’re simultaneously saying that cache is preferred and also discouraged… -Raymond]
  28. Anonymous says:

    Spake Raymond> “A cache of everything”? How much of “everything” is “everything”? Remember that Explorer is a combination of multiple components, many of which aren’t written by Microsoft

    Spake Raymond> You’re the one who proposed a cache, not me. Are you saying that if the user asks for a thumbnail of a 2MB image on the network, we should cache the entire file in memory? Yet you also say it’s rude to fill memory with these cached files. I’m confused. You’re simultaneously saying that cache is preferred and also discouraged.

    What they (and I) and I are advocating is commonly known as a buffer cache, it would appear. This lives in the kernel and uses whatever memory isn’t used for some other thing. That means that you do cache the whole 2M image if you can find space. If you care to argue about caching policy, take a look at the discussions centered around buffer caches, as the arguments you’re making have already been discussed for the most part.

    [Oh, you’re just talking about file system cacheing. I thought you wanted an application-level cache. Note that for cache coherency reasons, network files cannot be cached once the handle is closed. -Raymond]
  29. Centaur says:

    I was talking about an application-level cache for things that are reasonable to cache, like file names, attributes and WIN32_FIND_DATA. For file content, I argue for gracefully degrading Explorer functionality when the file in question is not local — by disabling all implicit behavior that requires opening the remote file.

    There already is a mode like this in Internet Explorer, when you uncheck “Show pictures” in Internet Options | Advanced | Multimedia. The user gets placeholders in place of images, but can request that an image be retrieved by right-clicking and selecting “Show image”. The image is then cached on the local disk, until it expires, is explicitly redownloaded, or is replaced by other cached files.

  30. Anonymous says:

    I would prefer an fs level cache, but since I don’t have that sort of sway with MS, I’ll settle for caching File data at the app level.

  31. Anonymous says:

    I the case the original article is referring to, my post was about eliminating automatic retries. The problem isn’t with a slow network; the problem is that programs know networks are unreliable and therefore automatically retry operations N times.

    Although it wasn’t clear originally, this slow (because it’s retried 50 times) operation isn’t retried 15 times, it’s the same operation done in 15 places. A cache wouldn’t actually help for this, though, because there’s no data to cache. If there was data to cache, the network operation would return immediately and we wouldn’t be complaining about automatic retries.

    All you could do at this point is cache the failure, which means that the other 14 operations will immediately fail. The problem then is how long do you cache the fact that you couldn’t access that network resource?

    DNS has this same problem. A DNS query which resolved successfully will be cached and will return quickly for some time in the future. A query which fails to resolve has nothing to cache, and you have to wait for it to timeout every single time.

  32. Dean Harding says:

    I would prefer an fs level cache, but since I don’t have that sort

    > of sway with MS, I’ll settle for caching File data at the app level.

    There already is a file system cache. Your computer would be unusable without it. However, for a general-purpose file system cache, you can’t cache data from the network longer than the file handle is open for anyway, because the system has no way to know when the file’s data changed otherwise (meaning it won’t be able to keep the cache coherent).

    Now, an application-level cache might be able to say "I don’t care if the data has changed on the server since the last time I accessed this file" (which it most certainly WOULD say if you’re getting various attributes over-and-over while populating the explorer view or a popup menu.

    The problem with an application-level cache is that you need it to work across multiple components, most of which are NOT controlled by Microsoft. Because it’s application-level, that means you need to put some hooks for those components to call instead of the regular system APIs, and then you can’t force the component developers not to use the system APIs anyway.

  33. Anonymous says:

    Wednesday, September 06, 2006 8:02 AM by TCLIU

    > Norman Diamond, Nekto2: Estimating the

    > runtime of a function based on analysis of

    > its code is possible, but brutally difficult.

    In order to prove that a problem reduces to the halting problem, it doesn’t matter how brutally difficult some hypothetical tool might be.  You still have to prove that all such hypothetical tools leave the problem unanswered.

  34. Dean Harding says:

    You still have to prove that all such hypothetical tools leave the problem unanswered.

    The halting problem is not always unanswerable. It’s only certain programs for which you can’t answer it.

    GetExecTimeEstimate is probably not in exactly the same class, because you only need an "estimate" (e.g. "short", "medium" and "long" are probably good enough answers in this case) but the cost of implmenting it (certainly in the general case) is probably far outweight by it’s actual usefulness.

  35. Dean Harding says:

    cost of implmenting it is probably far outweight [sic] by it’s actual usefulness.

    Oops, I meant the other way around…

  36. Anonymous says:

    Norman Diamond, Nekto2: Estimating the

    > runtime of a function based on analysis of

    > its code is possible, but brutally difficult.

    Ops.. I have not propose to implemennt universal function which could measure the time of any function execution. :) (Still some solution whould be to gather statistics – profiling, but that is not universal solution either).

    I just propose to implement special function which will help in case of measuring some set of operations. The main is – file handling operations. Even is shoud return 3 values – fist is minimum time it could take for the given parameters (say for file on network N: drive which is not in cache it never could be less then 1ms), second – maximum time (which includes firing all the timeouts there and the third is most likely time.  This values could not be calculated from code or any utomatic way . I should be calculated manually. Still this time may be inaccurate. The reason is to know is it 1ms or 30minutes :)

    Yet another solution (do not know is it possible) to create independent threads for operations which could hang or slow and update info on screen every 100ms with new data avaliable from all those threads. This should be used for calling extensions.

    The fist solutions will help developer to rewrite software to be aware of possible problems, the second will deal with software which are not so nice to implemet that awareness ;)

    ps. The icons in start menu is drawn that way AFAIK. On slow computers you see default icons the fist time you open it and then those redrawnd with custom icons.

    [The “maximum time” will nearly always be approximately INFINITE. You read from a file but, oops, a hard I/O error occurred and the kernel prompts the user to remove the CD, clean it, and reinsert it. The “do everything on a background thread” theory creates compatibility problems for extensions that assume they’re running on the main thread. -Raymond]
  37. Anonymous says:

    The “maximum time” will nearly always be approximately INFINITE.

    It for general case. But the reason for this all is – OS knows more then app should. It knows that there are CD and network and HDDs. And it knows of the HW state – is a cable pluged out or is a CD in drive. So if you call the:
    struct {long min, max, expected;} est;
    est=fopen_estimate(“d:fldfile”,”r”);

    You could get the INFINIT for CD and some reasonable 2 minutes for HDD (response or error will be returned before 2min). And still the program will be correct in the future if some new device or feature (say – “ask user if he allows the program to open the file on HDD in his home dir”) will be created.
    At now you could detect type of storage (cd/hdd/net), but you have no idea about future possible types there would be. Still the only thing you need to know is the one – could you access it freely to please a user with nice icons or better not to try because timeouts could made him angry. :)
    Or should you try to use N: for temp files if the minimum response time for file operations is 5 second (some webdav disk with VPN access on opposit side of Earth). MSWord tends to create temp file at the same location as original file is.

    > compatibility problems for extensions
    I know that is the reason, but for popular apps such as “WinZIP” every time new OS is out the new version of app is out also ;) Why not to allow them implement better interface.

    [Even for hard drives, the worst case is INFINITE. In the time between you call GetTimeEstimate and you actually access the file, the file is archived to tape. When you access the file, a message appears in the data center asking for the tape to be mounted. This can take a long time if the data center is closed for the holidays. Or the hard drive might encounter a bad sector and hang. -Raymond]
  38. Anonymous says:

    :)))

    Thanks for clarification and nice example.

  39. Anonymous says:

    Norman Diamond> However, for a general-purpose file system cache, you can’t cache data from the network longer than the file handle is open for anyway, because the system has no way to know when the file’s data changed otherwise (meaning it won’t be able to keep the cache coherent).

    Sure you can. Keep the file data in a buffer and return it the next time someone wants it. Since this is a local disk, you can control taht completely. If it’s a network drive, then things get complicated; you can probably cache a lot of metadata for up to a second or two. Dunno about the data. I generally don’t rely on network FSes for any sort of multiple access anyway, since I don’t trust it.

    > The problem with an application-level cache is that you need it to work across multiple components, most of which are NOT controlled by Microsoft.

    The app-level cache is a hack to get around the suboptimal behavior with network drives (in this case, it may only cache metadata). If some component developer bypasses it, then their stuff is slower, that’s all.

    > you can’t force the component developers not to use the system APIs anyway.

    Heh, sure you can. They don’t have any choice.

    [Not sure why you introduced local disks; the remark was regarding network drives. Caching network data without an open handle can give database programs (who are real sticklers for data consistency – go figure) the heebie-jeebies. -Raymond]
  40. Anonymous says:

    Not sure why you introduced local disks

    If it’s an OS level cache, then local disks are a given, and are handled differently from network disks.

    > Caching network data without an open handle can give database programs

    I admitted this in my comment, but if you run a database over a network link, you deserve what you get.

    [I didn’t think there was any concern over the local machine caching its local hard drives. It was the network case that was the issue. So you’re saying that people who rely on cache coherency over the network deserve what they get when it breaks? Why do we bother with file locking again? -Raymond]
  41. Anonymous says:

    So you’re saying that people who rely on cache coherency over the network deserve what they get when it breaks?

    No, running a database over the network is sinful , and will be punished. Network filesystems are a balance between acceptable performance and consistency. I don’t expect miracles and design apps accordingly.

    [How is “sinful and will be punished” different from “they deserve what they get”? -Raymond]
  42. Anonymous says:

    Simple: sinful and will be punished means that running a DB with its datafiles on a network share is fundamentally broken and will, err, break. You deserve what you get means that I will have no sympathy, nor will I pretend to be surprised.

    Just a bit of hyperbole to ease the tedium of breaking software.

    [Is it possible for something to be ‘sinful and will be punished’ while not ‘deserving what they get’? I’m just confused by that ‘No’ two comments ago. -Raymond]
  43. Anonymous says:

    Why yes, glad you asked ;)

    Sinful and will be punished: you did something bad and it’ll come back to you.

    Deserve what you get: you should’ve known better.

    Two aspects of the same thing, really. Of course, my definition of sin isn’t tied to morality, as I don’t consider myself any sort of paragon.

    [Okay, so then by ‘No’ you meant ‘Yes’. Now I’m not confused. -Raymond]
  44. Anonymous says:

    To clarify the database vs. cache coherency comment, databases are different animals from most things, especially when talking about consistency. A db will stress any sort of protocol well beyond what you can expect other applications to do. Expecting cache coherency in a windows share is bad, but you’ll mostly get away with it. Do that with a db and you will pay.

  45. Dean Harding says:

    Cooney: I think you’re thinking of SQL Server or Oracle here. Those sort of programs will simply refuse to run off a network share.

    But "Access" is also a database. Many people open Access *.mdb files off network shares, and Access relies on file locking to provide coherency (and it works, too, mostly)

  46. Anonymous says:

    Windows X11

    Actually "X Window System". There is no "s" on the end of "window" ;)

    http://en.wikipedia.org/wiki/X_Window_System

  47. Anonymous says:

    Thursday, September 07, 2006 6:02 AM by Dean Harding

    > The halting problem is not always

    > unanswerable.

    I was about to answer starting with "True, but …" but then noticed that your used exactly the wording that makes your statement false.

    Indeed there are halting problems for other classes of machines besides Turing machines.  But "The" halting problem means the halting problem for Turing machines.[*]  If you’re talking about a different halting problem then you have to specify which halting problem you’re talking about.

    And you didn’t say "Some" halting problems are not always unanswerable, you said "The" halting problem.

    [* It’s sort of like Windows, if you just say "Windows" then everyone knows you’re not talking about Windows X11 or Windows that are sold by Hardware stores (for sufficiently antique definitions of Hardware).]

  48. Anonymous says:

    Many people open Access *.mdb files off

    >network shares, and Access relies on file

    >locking to provide coherency (and it works,

    >too, mostly)

    If by "works…mostly" you mean that it corrupts the database on a regular basis, especially when notebook computers are unplugged from the network and reconnected while Access (or other applicaton using the Jet ODBC component) is open in the background, requiring that everyone be told to exit the program so that the database can be repaired, then yes, that’s true.

    However, I don’t think that’s a meaning of "works…mostly" that most people would recognize.

  49. Anonymous says:

    > If by "works…mostly" you mean that it corrupts the database on a regular

    > basis…requiring that everyone be told to exit the program so that the

    > database can be repaired, then yes, that’s true.

    Ah, my job description at my previous place of employment. Manifest tracking with an Access 97 database shared over a LAN and VPN via dial-up connections. I will say that at least Access was *usually* able to repair the database.

    Move to a SQL server? Gods man, we’re not made of money here! It doesn’t matter that we’ve lost an entire day’s worth of entering and modifying shipping manifests every two weeks for the last 5 years!

    Oh, the fun never ceased.

  50. Anonymous says:

    Funny how programmers tend to fix the wrong thing and to expect that OS cover for them. I would like to see all improperly written applications crash and burn until they get fixed or replaced by better written ones.

    It would be nice to have an API like this:

    QueryFileInformation(hFile, lpStruct);

    lpStruct should point to a structure with fields like name, size, attributes, thumbnail bitmap, ID3 tag, x and y size of an image, length of a song in seconds, video bitrate, document title, md5 checksum, etc, and it should be extensible. It should also have flags which you set if you are interested in particular piece of information. OS could then collect and return all that info by accessing the file only once.

    [That doesn’t fix anything. Fifteen functions call this new QueryFileInformation function, so the disk is accessed 15 times. You’re back where you started. -Raymond]
  51. Anonymous says:

    Fifteen functions call this new QueryFileInformation function, so the disk is accessed 15 times.

    But why OS is not marking this disk as “inaccessible”. That means – why it not caches negative answer? Then only 1st call will wait and next 14 to other files on same device (disk) would return immediately with same error. There should be the reason? ;)

    [As noted in the linked article, the file is on a network drive. Accessing a network drive means sending network packets around the world. And you can’t cache network data without an open handle that denies write. -Raymond]

    Thus repeat count will transform to wait timer – “how much to wait before retry”. So any call to same device will get cached error until that time alapsed. And if there would be new call after timer expired then again it will wait (but only the fisrt one (actually thouse wich are in between 1st_call_start to 1st_call_retry_timeout)).

    This cache time should be selected on basis of expected UI response. How much user could wait after he detects failure and fix it. It should be something from 1sec to 5 sec. (For example OS detects that Ethernet cable is pluged-in in ~2-3 sec and it is normal).

    ps. I know that if HDD has permanent errors it will be removed from list of devices attached (I have seen this on some computer when network share periodically dissapear untill restart). But I do not know exatly when :)

Comments are closed.