One of the extensibility points we have is the ability to specify third-party tools for diff (file comparison) and merge (three-way merge specifically) operations. For merge, when we call the third-party tools and then they return, we have to figure out whether we should accept whether we should trust the resulting file they produced (if it produced a file).
Initially we coded it so that if 1) the file was produced 2) the produced file was not empty (admittedly there are some scenarios where an empty output file is valid from a three-way merge, but that’s a very tiny corner case for us) and 3) the exit code was 0, then we accepted the output file.
However, we found at least one tool that would exit with a non-zero exit code even when the user had successfully resolved all the conflicts and saved the merged output, so we modified the code so that criteria #3 became a prompt to the user as to whether they want to accept the merged output file from the tool or not (the default tool we ship with, BTW, isn’t treated as special, it’s just the default merge tool, but is still treated just like any other third-party merge tool, and is called through the same code and same interface).
Unfortunately, this has now led to a “double prompt” scenario, in that if you resolve all your changes and exit the tool, you’re prompted once as you exit the tool (ours included) as to whether you want to save the output merged (this is typical as you exit most apps, of course), but then you get prompted again as we see the tool exited and prompt you for whether you want to keep the output file that the tool produced.
So, now we’re considering just prompting when the tool exits with a non-zero exit code. Since the typical case is that if the tool ran fine, the exit code was 0, this helps take care of the typical case by reducing a prompt, but also allows tools that return non-zero exit codes even when they run fine to have the user keep the output file.
What do you think? What’s approach would you like to see used? Keep in mind we have to code something that will work for not only the current set of merge tools that exist, but ones that may be written after our release.