I ran into a strange credential prompting scenario in SharePoint 2010 that took several hours to resolve. Hopefully, this post can save you some time.
We are upgrading a customer's MOSS 2007 environment to SharePoint 2010. The environment will retain the 2007 look/feel for a short period of time. Testing was going well until recently users started getting prompted for credentials when performing certain actions in SharePoint. What was strange is that the prompts typically appeared after the page had loaded and we could continue our work by simply canceling the prompt.
Our first thought was that there was a misconfigured WFE in the farm, so we ruled that out. We also ruled out client workstation configuration because the behavior was not occurring in one of our other environments that contained the same upgraded content.
The "Browser File Handling" setting (web application general settings) defaults to "Strict", so we set that to "Permissive". This resolved part of the problem and allowed us to open .html pages. However, we still were getting prompted during routine SharePoint functions.
The problem turned out to be a reference in a CSS file to an image file that no longer existed. The CSS entry was only applied in specific scenarios (ours was specific to the toolbar control), hence the reason that we were only prompted on specific SharePoint actions. Not sure what setting causes the prompting behavior for an invalid file entry, but now you know what to look for.