XAML build service fails with: “The type initializer for LibGit2Sharp.Core.NativeMethods' threw an exception”
I recently worked on a case where the TFS XAML build service was failing with the below exception:
The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception.
The setup included a TFS 2015 server and a separate machine running the XAML build service. The issue was noticed when the builds began to get stuck in the queue, and running for ever. Upon checking the event viewer, we could find some exception. The complete stack is given below:
Exception Message: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. (type TypeInitializationException)
Exception Stack Trace: at LibGit2Sharp.Core.NativeMethods.git_transport_register(String prefix, IntPtr transport_cb, IntPtr payload)
at LibGit2Sharp.Core.Proxy.git_transport_register(String prefix, IntPtr transport_cb, IntPtr param)
at LibGit2Sharp.GlobalSettings.RegisterSmartSubtransport[T](String scheme)
at Microsoft.TeamFoundation.Build.Hosting.BuildMachine.EndStart(IAsyncResult result)
at Microsoft.TeamFoundation.Build.Machine.WindowsService.EndStartMachine(IAsyncResult result)
Inner Exception Details:
Exception Message: l. (Exception from HRESULT: 0x8007007E) (type DllNotFoundException)
Exception Stack Trace: at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
at LibGit2Sharp.Core.NativeMethods.LibraryLifetimeObject..ctor()
at LibGit2Sharp.Core.NativeMethods..cctor()
Application: TFSBuildServiceHost.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.DllNotFoundException
at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
at LibGit2Sharp.Core.NativeMethods+LibraryLifetimeObject..ctor()
at LibGit2Sharp.Core.NativeMethods..cctor()
Exception Info: System.TypeInitializationException
at LibGit2Sharp.Core.NativeMethods.RemoveHandle()
at LibGit2Sharp.Core.NativeMethods+LibraryLifetimeObject.Finalize()
Faulting application name: TFSBuildServiceHost.exe, version: 14.95.25229.0, time stamp: 0x5723ce7c
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17031, time stamp: 0x53089862
Exception code: 0xe0434352
Fault offset: 0x00000000000068d8
Faulting process id: 0x1414
Faulting application start time: 0x01d22ee33b552588
Faulting application path: C:\Program Files\Microsoft Team Foundation Server 14.0\Tools\TFSBuildServiceHost.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 3770eb55-9ada-11e6-80c2-00155d00c926
Faulting package full name:
Faulting package-relative application ID:
Exception Message: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. (type TypeInitializationException)
Exception Stack Trace: at LibGit2Sharp.Core.NativeMethods.git_transport_register(String prefix, IntPtr transport_cb, IntPtr payload)
at LibGit2Sharp.Core.Proxy.git_transport_register(String prefix, IntPtr transport_cb, IntPtr param)
at LibGit2Sharp.GlobalSettings.RegisterSmartSubtransport[T](String scheme)
at Microsoft.TeamFoundation.Build.Hosting.BuildMachine.EndStart(IAsyncResult result)
at Microsoft.TeamFoundation.Build.Machine.WindowsService.EndStartMachine(IAsyncResult result)
Inner Exception Details:
Exception Message: Unable to load DLL 'git2-msvstfs': The specified module could not be found. (Exception from HRESULT: 0x8007007E) (type DllNotFoundException)
Exception Stack Trace: at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
at LibGit2Sharp.Core.NativeMethods.LibraryLifetimeObject..ctor()
at LibGit2Sharp.Core.NativeMethods..cctor()
We tried reconfiguring the build service as well. After reconfiguring, the service would start normally,
However, the error message was being logged in the event viewer, followed by the service start.
You could find some scenarios where this could happen, if you search online, but this case was unique. After spending some time troubleshooting, we were able to narrow it down. Below is the explanation:
It was the very useful Process Monitor tool that came to the rescue here as well. Since this was a case of assembly load failure, we enabled process monitor and reproduced the issue by stopping and starting the service, so that we can collect the details of the assemblies being loaded, and the paths that were being searched, for the assembly that failed to load.
Upon analyzing the logs with the help of one my colleagues, and comparing it against a sample process monitor collected from a working build machine at my end, we found that the assembly that was failing to load, git2-msvstfs is located and is getting loaded from:
C:\Program Files\Microsoft Team Foundation Server 14.0\Tools\NativeBinaries\amd64
However, from the process monitor log collected from customer’s environment, we found that the LibGit2Sharp.dll (which in turn calls the git2-msvstfs dll, is deployed to the GAC, and is being loaded from the GAC itself.
As a result, the build service host was only searching for the git2-msvstfs dll as well, in the GAC (and not in C:\Program Files\Microsoft Team Foundation Server 14.0\Tools\NativeBinaries\amd64).
Normally, LibGit2Sharp.dll is not deployed to GAC by during TFS build configuration. Our assumption is that a third-party application installed on the machine could have deployed this to the GAC.
To resolve the issue, we removed the LibGit2Sharp.dll from GAC by following the below steps:
- Navigate to C:\Windows\Microsoft.NET\assembly\GAC_MSIL
- Copy and paste LibGit2Sharp folder to a temporary location (to restore at a later point if needed)
- Open a Visual Studio Command prompt window as administrator.
- Run “gacutil -u LibGit2Sharp”
Once the assembly was uninstalled from GAC, we were able to start the build service successfully without errors, and the builds were running fine.
Please note that as mentioned, the dll would have been deployed to GAC by another application, and removing it might cause the application to malfunction. Please run with caution.
Hope this helped!
Content: Sreeraj RajendranReview: Romit Gulati