“Current thread must be set to single thread apartment (STA) mode before OLE calls can be made”

While calling a OpenFileDialog (https://msdn.microsoft.com/en-us/library/system.windows.forms.openfiledialog.aspx) via Installer class as a Custom Action in the Visual Studio Setup project, it just hangs there. When we attach a debugger we receive an error: "Current thread must be set to single thread apartment (STA) mode before OLE calls can be made. Ensure that your Main function has STAThreadAttribute marked on it. This exception is only raised if a debugger is attached to the process."  

The same code works fine on Win XP and 2003. It only occurs on Win Vista and higher version of Operating Systems. The sample code is mentioned below: 

            MessageBox.Show("Cannot Locate Config File");

            MessageBox.Show("Pid : " + System.Diagnostics.Process.GetCurrentProcess().Id.ToString());

            String Location = String.Empty;

            OpenFileDialog frm = new OpenFileDialog();  

            frm.InitializeLifetimeService();

            frm.Filter = "Config Files (*.config)|*.config| (*.xml)|*.xml";

            frm.Title = "Browse Config file";

            DialogResult ret = frm.ShowDialog();

            if (ret == DialogResult.OK)

            Location = frm.FileName;

            InitializeComponent();

The problem is that the MSI thread is running as an MTA thread, but the FileDialog.ShowDialog requires an STA thread.  To achieve this you will need to start a STA background thread and call the dialog from that thread. Basically I did the following:

-          Added the DialogState class.  This keeps track of the input and output for the thread.

-          Added the STAShowDialog function.  This function takes a FileDialog, calls ShowDialog on a background STA thread, and then returns the results.

-          Changed the call from DialogResult ret = frm.ShowDialog(); to DialogResult ret = STAShowDialog(frm);

You may need to add exception handling separately. 

Here is the complete code:

using System;

using System.Collections;

using System.Collections.Generic;

using System.ComponentModel;

using System.Configuration.Install;

using System.Linq;

using System.Windows.Forms;

namespace WindowsFormsApplication1

{

    [RunInstaller(true)]

    public partial class Installer1 : Installer

    {

        public Installer1()

        {

            MessageBox.Show("Cannot Locate Config File");

            MessageBox.Show("Pid : " + System.Diagnostics.Process.GetCurrentProcess().Id.ToString());

            String Location = String.Empty;

             OpenFileDialog frm = new OpenFileDialog();

             frm.InitializeLifetimeService();

            frm.Filter = "Config Files (*.config)|*.config| (*.xml)|*.xml";

            frm.Title = "Browse Config file";

            DialogResult ret = STAShowDialog(frm);

            if (ret == DialogResult.OK)

                Location = frm.FileName;

             MessageBox.Show(Location);

             InitializeComponent();

        }

        /* STAShowDialog takes a FileDialog and shows it on a background STA thread and returns the results.

         * Usage:

         *   OpenFileDialog d = new OpenFileDialog();

         *   DialogResult ret = STAShowDialog(d);

         *   if (ret == DialogResult.OK)

         *      MessageBox.Show(d.FileName);

         */

        private DialogResult STAShowDialog(FileDialog dialog)

        {

            DialogState state = new DialogState();

            state.dialog = dialog;

            System.Threading.Thread t = new System.Threading.Thread(state.ThreadProcShowDialog);

            t.SetApartmentState(System.Threading.ApartmentState.STA);

            t.Start();

            t.Join();

            return state.result;

        }

    }

    /* Helper class to hold state and return value in order to call FileDialog.ShowDialog on a background thread.

     * Usage:

     *   DialogState state = new DialogState();

     *   state.dialog = // <any class that derives from FileDialog>

     *   System.Threading.Thread t = new System.Threading.Thread(state.ThreadProcShowDialog);

     *   t.SetApartmentState(System.Threading.ApartmentState.STA);

     *   t.Start();

     *   t.Join();

     *   return state.result;

     */

    public class DialogState

    {

        public DialogResult result;

        public FileDialog dialog;

        public void ThreadProcShowDialog()

        {

            result = dialog.ShowDialog();

        }

    }

}