When you try to create a new setup project/open an existing setup project, Visual Studio 2005/2008 IDE may hang


To work around this problem, run DCOMCNFG.exe and clear the check box for the Provide additional security for reference tracking option on the Default Properties tab.

1. Click Start, point to Programs, point to Administrative tools, and then click Component Services.
2. When the Component Services explorer is open, double-click Component Services, and then click Computers to open the Computers folder.
3. Right-click My Computer, and then click Properties.
4. Click the Default Properties tab.
5. Disable the Provide additional security for reference tracking check box near the bottom of the page
6. Click OK.

Repro Steps
==========
1. Click Start, point to Programs, point to Administrative tools, and then click Component Services.
2. When the Component Services explorer is open, double-click Component Services, and then click Computers to open the Computers folder.
3. Right-click My Computer, and then click Properties.
4. Click the Default Properties tab.
5. Enable the Provide additional security for reference tracking check box near the bottom of the page.
6. Click OK.
Create a new setup project/open an existing setup project in Visual Studio 2005/2008. Note Visual Studio IDE hangs

More detail Information
====================

From the hang dump:
The first thread waits till the other thread finishes. Thread 0 is apartment threaded.

0:000> kL
ChildEBP RetAddr
000fff74 7c90e9c0 ntdll!KiFastSystemCallRet
000fff78 7c8025cb ntdll!ZwWaitForSingleObject+0xc
000fffdc 5ccbb98d kernel32!WaitForSingleObjectEx+0xa8
00100004 5cc6ca93 dpdpl!CVsThreadBase::Start_base+0x64
0010003c 5cd8cdad dpdpl!CVsdDeployable::Validate+0xcb
00100078 5cd946f4 dpprj!CVsdPrjBaseProject<IVsdPrjSetupProject,CVsdPrjSetupProject,&CLSID_VsdPrjSetupProject>::StartBackgroundValidation+0xa3
001000b0 501024ba dpprj!CVsdPrjRootNode<&CLSID_VsdPrjCabProject>::OnAfterOpenProject+0x109

The other thread is trying to unmarshal an apartment threaded COM object, which was created in thread 0. So it waits on thread 0 to process windows messages and deadlock

0:020> kL
ChildEBP RetAddr
0892f670 7c90e9c0 ntdll!KiFastSystemCallRet
0892f674 7c8025cb ntdll!ZwWaitForSingleObject+0xc
0892f6d8 7c802532 kernel32!WaitForSingleObjectEx+0xa8
0892f6ec 7752abbd kernel32!WaitForSingleObject+0x12
0892f708 77601e51 ole32!GetToSTA+0x6f
0892f728 7760109a ole32!CRpcChannelBuffer::SwitchAptAndDispatchCall+0xf6
0892f808 7751047c ole32!CRpcChannelBuffer::SendReceive2+0xb9
0892f874 77510414 ole32!CAptRpcChnl::SendReceive+0xab
0892f8c8 77ef3db5 ole32!CCtxComChnl::SendReceive+0x113
0892f8e4 77ef3eac rpcrt4!NdrProxySendReceive+0x43
0892fcc0 77ef3e42 rpcrt4!NdrClientCall2+0x1fa
0892fce0 77e89aa4 rpcrt4!ObjectStublessClient+0x8b
0892fcf0 77511f50 rpcrt4!ObjectStubless+0xf
0892fd40 77511fa3 ole32!CStdMarshal::RemoteAddRef+0xb3
0892fd64 775117d3 ole32!CStdMarshal::GetNeededRefs+0x4a
0892fd98 7751168a ole32!CStdMarshal::ConnectCliIPIDEntry+0xa0
0892fdc4 77510cb0 ole32!CStdMarshal::MakeCliIPIDEntry+0x8c
0892fdf0 77510be0 ole32!CStdMarshal::UnmarshalIPID+0x66
0892fe44 77510fb1 ole32!CStdMarshal::UnmarshalObjRef+0xef
0892fe5c 77510f71 ole32!UnmarshalSwitch+0x25
0892fe90 77510fd9 ole32!UnmarshalObjRef+0x7f
0892ff14 77556e24 ole32!CoUnmarshalInterface+0xa7
0892ff30 5ccbb8b5 ole32!wCoGetInterfaceAndReleaseStream+0x16
0892ff44 5ccbba16 dpdpl!CVsThreadBase::UnmarshalParentIUnknown+0x1a
0892ff74 781329bb dpdpl!CVsThreadBase::InternalThreadMain+0x4c
0892ffac 78132a47 msvcr80!_callthreadstartex+0x1b
0892ffb4 7c80b683 msvcr80!_threadstartex+0x66
0892ffec 00000000 kernel32!BaseThreadStart+0x37

The COM is an internal COM object and so it doesn’t have registration information in the registry. Since it works on other machines my guess is that we unmarshal the interface before its marshaled. In order to compare it, I took a hang dump from a working machine with a break point on dpdpl!CVsThreadBase::UnmarshalParentIUnknown

I wanted to find why marshalling did not happen on working cases. To this effect I put breakpoints on all the functions in the secondary threads stack up to GetToSTA. Then I figured out where it deviated in the working machine. This led me to function ole32!CStdMarshal::GetNeededRefs. Checking this function lead me to a value and it was different between failing and working machines. The difference in values was a flag which according to the doc represents the above DCOM setting.

Reference: http://msdn2.microsoft.com/en-us/library/ms680051.aspx#_com_setting_machine_wide_reference_tracking

 


Comments (0)

Skip to main content