The psychology of confirmation, or something, I don't know what to call it


There is probably a name for this phenomenon. I will illustrate it below.

"Is there a way to configure the system to do X?"

Go to the Y dialog and select Z.

"It doesn't work."

I just tried it. It works for me. I'm using ⟨configuration details⟩.

"Thanks. It's working."

Comments (46)
  1. Joshua says:

    Last time I was able to investigate one of these, it was working fine and the screen being looked at was stale.

  2. Boris says:

    'Quick' error reporting? Analogous to a 'quick' format?

  3. Eric says:

    This seems pretty normal to me.  If I use an approach provided by a forum user and it doesn't work, I'll generally shoot back a "No-go, do I need to have anything else configured?" message and then go Googling for the answer myself.  If the forum gets back to me first, so much the better, but it's not uncommon for that last line to be "Thanks, I just found those settings (and some other tips) at http://www.linkforposterity.com and it's working."

  4. DaveK says:

    Users often tell you've they've tried something according to your instructions when they simply haven't at all.  This sounds like a way they can resolve the ensuing cognitive dissonance.

  5. Markus says:

    Well of course, no one wants to be a beta tester.

  6. Mark (The Other Mark) says:

    Maybe I'm in the minority here, but it's a minor pet peeve for me when people update with things like "Thanks, I just found those settings (and some other tips) at http://www.linkforposterity.com and it's working."

    Invariably, I come across the post six months later, and linkforposterity.com has done a site revamp and the link is now dead. If I'm lucky, Archive.org has it.

    I prefer to update with "Thanks, I just found that you need LtU&E set to 42 or higher to use Snap/Crackle over Pop (and some other tips) at http://www.linkforposterity.com and it's working. <Optional cut and paste of paragraph in question, if appropriate>"

  7. Goran says:

    Isn't this called "try it seriously this time"? :-)

  8. bzakharin says:

    Possibly related, I find that it is much more likely I find something if someone tells me where it is, even if I already looked there and didn't find it, be it a product in a store aisle or a food item in a particular place in the fridge

  9. Kevin says:

    Related: "It doesn't work" is probably the worst thing you can tell a developer, in any context.

  10. Dave Bacher says:

    Also related:

    If I'm standing over a user's shoulder watching, whatever problem they wanted me to see will never occur.

    Mysteriously, sending step-by-step instructions and even a screencast, the customer is unable to get something to work.  But if a developer is within 3m, the setting mysteriously has the intended effect.

  11. Jason Warren says:

    The act of forcing the user to go through the steps resolves the issue. Same too with hovering over their shoulder. It's like assembling those pieces of furniture that come with the multi-step instructions. Sometimes you glaze over a few steps and get stuck. Only the act of going back and retracing your steps will work, even if when doing so you (think you) did nothing different than before.

    In reality you missed a step. Go back and doing it correctly unsurprisingly resolves the issue.

  12. Anon says:

    @Mark (The Other Mark)

    That's why you'll get downvoted into oblivion on SO for failing to properly include the content of a link.

    Also, I loathe anyone who leaves a stupid response such as "I fixed it" with no detail.

    Also, https://xkcd.com/979/

  13. Devin says:

    @Anon

    Or an extension to that:

    1. Google a problem today and get unrelated results and/or clickbait results.

    2. Post question to SO or some other site.

    3. Google the problem tomorrow and get same unrelated result and/or clickbait PLUS your, as yet, unanswered post from yesterday.

  14. Brad says:

    Sometimes this is the result of the user inventing a feature (or the behavior of a feature) completely and assuming that checking the box in Dialog Y is what enables their imaginary feature. Then, when you explain to them what the option actually does (or they gather the actual behavior from the context of your explanation), they realize that it's working as intended.

  15. steven says:

    @Kevin

    Amen to that. It once took me 2 weeks of back-and-forth to figure out that when a Japanese customer said an RSS feed "stopped working", they meant the summaries in the feed had decreased from 175 to 125 characters. I could see the feed being requested by their system, could request the feed myself and see what to my eyes looked like a perfectly valid feed (and it was, just different to what they had received earlier), but the most I ever got out of them was that it "wasn't working".

  16. Eric Wallace says:

    I agree this phenomenon needs a good name.

    In my experience, there's always something left unsaid in that final "Thanks. It's working." statement -- many people are too proud to admit they missed a step the first time.

  17. Kirby FC says:

    — I just tried it. It works for me. I'm using ⟨configuration details⟩.

    "Thanks. It's working."

    Maybe the person was doing it right, but, wasn't using the same 'configuration details' that you mentioned. They changed their configuration settings and now it works. Seems simple and obvious to me. Which of course means I'm probably wrong.

    [My configuration details were something like "I'm using a 1GB ATI FirePro V" - not the sort of configuration detail somebody can change easily. -Raymond]
  18. Simeon Pilgrim says:

    I experience "it doesn't work" all the time, it seems almost like a form of Confirmation bias, where they have a mental model of the system (that is not working when they use it "correctly") so are just looking for others to confirm "it is broken" (like Google is down) verse they mis-understand how it's used, or what the expected results should be. They are not open to the idea, that their not following the instructions might be the cause of the issue, and also are not open to how such a mis-alignment might be resolved, thus do not frame the "issue" in a context that enable fast resolution of the difference of models.

    The few issue reports you get that are truly descriptive, of expectations, steps, and known starting state, usually demonstrate your talking to a fellow tester/developer.

  19. Danny says:

    @Dave Bacher - It's a well known fact, not just to IT world but to every other domain. Here are some 1st hand experiences from people I personally know and they said happened to them. My older brother is a electronic technician, with an experience that spans from '70 to today. Half the time was called to repair the client TV/Radio/Stereo etc, the defect was not there, the electronic device was working perfectly. Most times was called back with the message: "after 5 minutes you walked out the door the defect reappeared". One of my friends is also with 30+ years experience in his field, he is a plummer. Like my brother, half the times was called the pipe/valve/toilet bowl etc was working like a charm. Plenty of times same call: "5 minutes after you left was not working again". Another friend is a car technician. Same deal -> goes to client -> car is working perfectly -> leave -> car is broken again within the same day. Needles to say I experienced this within IT in my 20+ years of dealing with computers, both in support and development. It's like the PC/TV/pipe/car has a life of its own, and once a very competent person is nearby realizes is futile to misbehave. I do believe this is something eventually the quantum mechanics will figure it out because definitely there is something more and our actions from a far does have reactions at levels we still don't understand or realize.

  20. cheong00 says:

    @Steven Don: That's why most of the modern support ticket system have "expected behavior" field, that make them state clearly what they think is "wrong".

    Btw, it still bug me some time when I see "It should work correctly" on the field even when we made it mandatory.

  21. Joshua says:

    Work correctly or similar I'm been known to put for expected behavior when the bad behavior starts with "Crash when ..."

  22. cheong00 says:

    @Danny: My brother's computer cannot boot (no screen, no trace of POST action sound, no hardware beep, fans spinning). When we took it to computer shop, it works perfectly. After we take the computer back home, it booted successfully for a few months then fails with the same symptoms again. Then it works again after we took it to computer shop, no part is changed during that.

    The technician is helpful enough to let us replace anything except the harddisk (power, RAM, motherboard, CPU, etc.) once at a time but the problem keeps recurring. When his computer dies again, we joke that "There's nothing wrong in there, his computer just want to take a walk to the shop again".

  23. Zan Lynx' says:

    @cheong00: Sometimes there isn't any more to say about how it should work. A bug report for a hypthetical "service" command. If I type "service httpd start" then I expect it to start the httpd service. A shorter way of saying that is "It should work correctly."

    The hard part is always identifying the meaning of "correct" or even in my example, "httpd". If the httpd service is aliased to nginx when it was apache last week, I can see how the bug reporter would think that was a bug while the developer would think it's correct.

    This easily leads to both sides deciding the other is an idiot, since it's clear to the developer that this user didn't configure his nginx web server, while the user sees the obvious fact that it worked last week and doesn't work now, and doesn't realize an entirely different "httpd" is running. Double points if neither the dev working on the bug or the user realize someone else changed the httpd alias.

  24. Simon says:

    Yeah, it's pretty familiar. The tester can reproduce a problem 100% of the time - until he calls the developer over, at which point the problem vanishes and is never seen again.

    Of course, the opposite also happens... long-standing bugs that went unnoticed until the tester found them, at which point they start occurring all the time.

  25. Cesar says:

    @cheong00: that's different. Your case sounds more like a bad solder joint or something like that; the movement and vibration of taking it to the computer shop could be enough to "fix" it temporarily.

    What @Danny and @Dave Bacher are talking about is the opposite; the gadget is not moved, instead the technician moves towards the gadget. The physical proximity of a knowledgeable person is the variable in these cases.

    @Simon: that's called a schroedinbug (catb.org/.../schroedinbug.html).

  26. Rick C says:

    @Danny, you know what's going on there, don't you?  The user has already forgotten the correct way to do it.

  27. Anon says:

    @cheong00

    I had a similar problem once. Main symptom was an erratic KB, but only sometimes, and replacing the KB didn't help. Moving my PC to a grounded outlet (found out the one I had it plugged into was the ONLY ungrounded outlet in my apartment after plugging in a new GFI surge suppressor), and every problem I had went away like magic.

  28. Anon says:

    @Joshua

    A crash, of any sort, should NEVER be As Designed. The program should log exceptions, but it should always gracefully handle them. Even if you HAVE to shut down because an exception places your app in an unknown state, you should always gracefully close (and provide whatever diagnostic details might be necessary to resolve the problem manually).

    The only time a hard crash is acceptable is when it occurs in code you can't access. And even that's avoidable most of the time.

  29. cheong00 says:

    @Zan Lynx': By quoting that, of course the unspoken part is the "problem description" field states the unhelpful "it doesn't work" text.

    And btw, in your example of HTTPD unable to start, of course the "expected behavior" should be "the web server start successfully after the command is executed", not something like "It should work correctly."

    Of course we know there should be something went wrong if you have to submit bug report, but if you don't let us know what went wrong, there's nothing we could do except calling them by phnoe.

  30. meh says:

    Any deviation from expected behaviour is exciting. Especially if you can restore previous state afterwards.

  31. dave says:

    re @Anon

    What exactly is the difference between 'shutting down because an exception places your app in an unknown state' and 'crashing'?   I see none.  What we call (application) crashing is simply a system-provided handler that shuts down an application because an app has placed it in an unknown state.

    Are we merely talking about the distinction between displaying, say, a cloud of registers, and printing out 'Something unexpected happened but displaying this message is somehow friendlier' ?

  32. Kevin says:

    @Anon: If I'm writing a C library, and you call one of my functions with an invalid pointer, I will segfault.  I will not goof around with IsValidFooPtr().  Crashing by design is the Right Thing in this case.

  33. D-Coder says:

    Sometimes you can't understand something until you see it.

    Sometimes you can't see something until you understand it.

  34. Falcon says:

    @Rick C:

    It's possible, however, imagine if the problem is something like rough running at idle - if the car's systems are in good working order, there's nothing the user can do with the car's controls that could cause that. Breaking things by tampering is a different story...

  35. Anon says:

    @Kevin

    The caller is responsible for handling the exception. The library should just return the PROPER exception. A segfault which shows that the pointer is invalid is perfectly fine, but

    However, you should probably handle the case anyway, lest your library become known as "that garbage that can't return a proper exception to the caller and instead brings down the entire process."

    @dave

    There's a MASSIVE UX difference between your application hard-crashing, and handling an exception with a friendly message to the user. For one thing, if your application hard-crashes, you're giving up the ability to log a lot of the information you might need to solve the problem.

    [How do you "handle" the case of an invalid pointer? Do you become the library that "sometimes succeeds, sometimes fails, sometimes returns a proper exception, sometimes corrupts the stack"? -Raymond]
  36. Anon says:

    @Falcon

    There ARE things the user could do. First thing that comes to mind is that they could be resting a foot on the gas or brake, either of which could cause 'inexplicable' issues. Someone new in the car might not habitually put their feet or hands on the same controls.

  37. Bob says:

    I would say that this pattern happens because the user doesn't have a time machine. There is a high probability that the conversation will go as:

    "It doesn't work."

    — Oh...that's strange.  It should/used to work that way.

    or

    "It doesn't work."

    — I just tried it. It works for me.  All I had to do is add your project to the configuration database.

  38. chrisaverage says:

    I happen to experience similar thing often from a 3rd person perspective and it usually goes something like this:

    A: Stupid company X. They can't do configuration of Y right. Why can't I just do <imaginary procedure that makes no sense>?

    B: Have you tried steps 1,2,3 and 4?

    A: Yeah, I tried everything. It doesn't work. Stupid X.

    B: Can you show me what you did?

    A: <does 4 and semi-does 2> See? Why can't X do anything right?

    B: <silently does steps 1,2,3,4 and fixes the issue>

    A: Thanks. Stupid company X.

    Me: <giggles>

  39. cheong00 says:

    Oh. On related note, I think Joel's blog these days a lot less interesting to read than it was a few years ago.

  40. Scarlet Manuka says:

    @Devin:

    (on googling for a problem only to find your own unanswered question about it)

    We had an amusing variation of this recently. Opened a support ticket for a problem we were experiencing with a certain piece of software. They also opened discussions with us by email. While they were working on it, we came up with a workaround, which we posted to the support ticket. After another week or so, in the email discussions they said they'd found a potential solution - which turned out to be a verbatim quote of the workaround we'd posted to the support ticket.

  41. Variations of the second response:

    "Can you send a screenshot?"

    "What error message do you get?"

    "What happens instead?"

    This is why any good bug report contains both a description of what the user *expected* to happen, and what *actually* happened.

    http://www.joelonsoftware.com/.../fog0000000029.html

  42. Falcon says:

    @Anon:

    I would still put that down to an incomplete troubleshooting process or an obscure intermittent problem, rather than operator fault.

    I am not arguing against the phenomenon described in the article, though - I have experienced it from both sides!

  43. Anon says:

    @raymond

    You can't prevent 100% of problems, but you can do some basic checking (Is it null, is it outside a valid memory range, etc.)

    [What is the valid memory range? Is the stack a valid memory range? (If you say yes, then you are vulnerable to the same issue as IsBadXxxPtr. If not, then nobody can pass pointers to local variables, which is probably a style-cramping restriction.) What about a memory-mapped file, or a video framebuffer, or a commit-on-demand region? -Raymond]
  44. Joshua says:

    @Anon: The real reason is "is this a valid memory range" is too expensive. Checking for ptr < (T *)4096 is reasonable though.

    [The question is whether you should be hiding other people's mistakes or shining a light on them. Generally speaking, if there is no security boundary, you should just crash because you have proof that the system is in a corrupted state. Continuing to run just makes the corruption worse. -Raymond]
  45. Anon says:

    @raymond

    "A valid memory range" just means "not a flat-out impossible pointer."  Can I write to the first 8k of memory? 16k? 32k? 64k? No? Then a 'pointer' that small should be outright rejected as impossible. Do I have a maximum theoretical limit on my pointer range? Then I should reject any pointer higher than that.

    But you've missed the part where I indicate that you should log the exception and gracefully shut down.

    If you're writing a library, you should return an appropriate response. If the caller gives you the same bad pointer twice, after you've told them you got an exception accessing it the first time, that's hardly your fault. But you should, the first time, say "That's not right." Crashing inside your library without telling the caller is the equivalent of ripping the house down around you because someone asked for a pencil and you couldn't find one.

    "Crashing" means your application just disappears, leaving a cryptic (to them) error that most users have *no hope* of debugging, possibly no hope of reproducing, and worst of all - no hope of fixing.

    Logging it and shutting down (NOT just ignoring it) means your application disappears, but gives the user the opportunity to contact support with Useful information.

    In short: I'm not saying that you should eat the exception and continue as though nothing at all happened, nor am I saying that it is possible to validate literally every possible pointer.

    [If you just crash on the invalid pointer, then the app will take its normal "oh no, I crashed!" measures, be that logging a stack trace, recording a crash dump, etc. Since you are a library, you don't know what logging system the app uses, so where will you log it? Into some custom log that only your library uses? Then the app author will say "My users tell me that it crashes in this library, but it's not showing up in my crash logs." -Raymond]
  46. No'am says:

    I've always termed this the 'white coat syndrome'. It's a well known fact that seeing a white coat (a doctor's) causes one's blood pressure to increase. Similarly, when technical support is within 3m (as per Dave Bacher), whatever didn't work before will now work. I think that the physical presence of the technician causes the person to concentrate more on the task to be solved.

Comments are closed.

Skip to main content