Building Non-MSBuild Projects With Team Build


Building non-MSBuild projects in Team Build has never been a particularly nice experience…  You can make it work, typically by invoking DevEnv.exe or some other tool using an Exec task, but you don’t get any output in your build report, can’t control the output sent to your build log, etc.  Additionally, it was recently pointed out to me (by D. Omar Villareal from Notion Solutions) that when a build type is set up that only builds non-MSBuild projects, the resulting builds don’t end up in default Build Reports generated by TFS, since the relevant information doesn’t get generated for the Warehouse!

To solve some of these issues, I’ve put together a custom MSBuild task for compiling non-MSBuild projects in Team Build.  The basic idea here is to execute an external executable (like the Exec task does), redirect stdout and stderr to the build log, and insert some minimal data into the Team Build database.  I had it inherit from the task base class I put together way back in August of last year.  As always, no guarantees are made about the awesomeness, or lack thereof, of this sample code.

using System;
using System.Web.Services;
using Microsoft.Build.Framework;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Proxy;
using System.Diagnostics;

namespace CustomTasks
{
    public class NonMsBuildProject : TeamBuildTask
    {
        #region Properties

        [Required]
        public String ToolPath
        {
            get
            {
                return m_toolPath;
            }
            set
            {
                m_toolPath = value;
            }
        }

        [Required]
        public String ProjectName
        {
            get
            {
                return m_projectName;
            }
            set
            {
                m_projectName = value;
            }
        }

        public String Platform
        {
            get
            {
                return m_platform;
            }
            set
            {
                m_platform = value;
            }
        }

        public String Flavor
        {
            get
            {
                return m_flavor;
            }
            set
            {
                m_flavor = value;
            }
        }

        public String Arguments
        {
            get
            {
                return m_arguments;
            }
            set
            {
                m_arguments = value;
            }
        }

        public String WorkingDirectory
        {
            get
            {
                return m_workingDirectory;
            }
            set
            {
                m_workingDirectory = value;
            }
        }

        [Output]
        public int ExitCode
        {
            get
            {
                return m_exitCode;
            }
        }

        #endregion

        #region TeamBuildTask Methods

        protected override bool ExecuteInternal()
        {
            // Create process to compile the non-MSBuild project.
            ProcessStartInfo psi = new ProcessStartInfo();
            psi.WorkingDirectory = WorkingDirectory;
            psi.FileName = ToolPath;
            psi.Arguments = Arguments;
            psi.UseShellExecute = false;
            psi.CreateNoWindow = true;
            psi.RedirectStandardError = true;
            psi.RedirectStandardOutput = true;

            Process p = new Process();
            p.StartInfo = psi;

            p.OutputDataReceived += new DataReceivedEventHandler(p_OutputDataReceived);
            p.ErrorDataReceived += new DataReceivedEventHandler(p_ErrorDataReceived);
            p.EnableRaisingEvents = true;

            BuildEngine.LogMessageEvent(new BuildMessageEventArgs(GetBuildStepMessage(), null, "NonMsBuildProject", MessageImportance.Normal));

            if (!p.Start())
            {
                return false;
            }

            p.BeginOutputReadLine();
            p.BeginErrorReadLine();

            do
            {
                p.WaitForExit(100);
                p.Refresh();
            }
            while (!p.HasExited);

            m_exitCode = p.ExitCode;

            // Create project details object and associated with the build.
            ProjectData projectData = new ProjectData();
            projectData.CompileErrors = m_errors;
            projectData.ProjectFile = ProjectName;
            projectData.PlatformName = Platform;
            projectData.FlavourName = Flavor;

            BuildStore.AddProjectDetailsForBuild(BuildUri, projectData);

            return (p.ExitCode == 0);
        }

        protected override string GetBuildStepName()
        {
            return "NonMsBuildProject";
        }

        protected override string GetBuildStepMessage()
        {
            return String.Format("Building a non-MSBuild project. Command-line: '{0} {1}'.", ToolPath, Arguments);
        }

        #endregion

        #region Private Members

        private void p_ErrorDataReceived(object sender, DataReceivedEventArgs e)
        {
            // Write errors to log file and increment compilation error count.
            if (e.Data != null &&
                !String.IsNullOrEmpty(e.Data.Trim()))
            {
                BuildEngine.LogMessageEvent(new BuildMessageEventArgs("Error: " + e.Data, null, "NonMsBuildProject", MessageImportance.High));
                m_errors++;
            }
        }

        private void p_OutputDataReceived(object sender, DataReceivedEventArgs e)
        {
            // Write stdout to log file.
            BuildEngine.LogMessageEvent(new BuildMessageEventArgs(e.Data, null, "NonMsBuildProject", MessageImportance.Normal));
        }

        private String m_toolPath;
        private String m_projectName;
        private String m_arguments;
        private String m_workingDirectory;
        private String m_flavor;
        private String m_platform;
        private int m_exitCode = 0;
        private int m_errors = 0;

        #endregion
    }
}

To use this task, do something like the following in your TfsBuild.proj file:

  <UsingTask AssemblyFile="CustomTasks.dll" TaskName="NonMsBuildProject" />
  
  <Target Name="AfterCompile">

    <NonMsBuildProject TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                       BuildUri="$(BuildUri)"
                       ToolPath="$(TeamBuildRefPath)\..\devenv.com"
                       Arguments="$(SolutionRoot)\VS2003Solution\VS2003Solution.sln /Build Debug|x86"
                       WorkingDirectory="$(SolutionRoot)\VS2003Solution\"
                       ProjectName="VS2003Solution"
                       Platform="x86"
                       Flavor="Debug" />                       

  </Target>

In the above example, you would still have some work to do getting the generated binaries copied to the appropriate locations, etc.  Hopefully this will be enough to get some folks started, however!


Comments (24)

  1. In my last post I mention that exchanged emails with some Microsoft guys and they explained to me why…

  2. Buck Hodges says:

    Aaron Hallberg wrote a great post today showing how to use a custom task to better integrate other build

  3. 一個常問的問題是, 什麼是 MSBuild ? , 什麼是 TeamBuild ? 簡單來說, MSBuild 才是微軟在 建構管理中的核心技術引擎, Team Build 則是 Team Foundation

  4. Martin Hinshelwood on TFS Admin Tool 1.2 Gotcha. Adam Singer on Can you direct me to Directory Services?…

  5. Many Visual Studio project types are not supported by MSBuild – setup projects, reporting projects, etc.

  6. Because many Visual Studio project types are not supported in MSBuild, many Team Build users end up needing

  7. Buck Hodges says:

    Aaron has written a great post on using Visual Studio (devenv) from within Team Build as part of the

  8. Maor David says:

    Building non-MSBuild projects is possible. For example you have to build VS2003, Installer, C++, Delphi

  9. AlexeyYumashin says:

    Hi, I used your code to create a team build task for VB6 projects. However, I can’t understand several things:

    1. In case compilation fails (due to some syntax errors in VB6 code, for example), the last Build Step looks like "Successfully completed" and the overall status of the "Buils steps" section is also "Successfully completed". Although debugging shows that p.ExitCode = 1. Maybe smth. is forgotten?
    2. If I examine the BuildLog.txt file, I see the following:

    Target CoreClean:

     Skipping target "CoreClean" because it has no outputs.

    AFAIU, it’s about cleaning the BuildDirectoryPath. If so, it’s not good that it isn’t cleaned, I guess. Can I fix this somehow?

    Thank you.

    PS: I’m using TFS 2005 and Visual Studio 2005 Team Suite.

  10. AlexeyYumashin says:

    And one question more:

    I created one more property in your custom task:

    private ITaskItem[] m_outputFiles;

    [Output]

    public ITaskItem[] OutputFiles

    {

       get { return m_outputFiles; }

    }

    After successfull compilation (in the ExecuteInternal function) I fill this array with the names of the binary files being just compiled. In case of VB6 there’s exactly one file – DLL or EXE or OCX. Debugging shows this code works fine. But then why the "Copying binaries to drop location" build step does nothing in such case? I can see only the "BuildLog.txt" in the DropLocation folder, and nothing more. This step IS NOT marked as FAILED, at the same time! What’s the secret?

    Thank you.

  11. AlexeyYumashin says:

    BTW, why is it necessary to use the AfterCompile target? isn’t it possible to override the CoreCompile target?

    1. In case compilation fails (due to some syntax errors in VB6 code, for example), the last Build Step looks like "Successfully completed"… Maybe smth. is forgotten?

    I would expect a non-zero exit code to fail the build, since the task will return false in this case (p.ExitCode == 0 will be false). Have a look at the build log – perhaps you have the task marked with ContinueOnError=true or something?

    1. CoreClean is typically called once per Platform/Flavor combination specified for your build type.  If you add entries to your PlatformToBuild item group (in tfsbuild.proj) for the platform(s) and flavor(s) you are building, the CoreClean target should get run.  It may, however, not do exactly what you would like – by default it just deletes the Sources, Binaries, and TestResults directories on the build machine.  Your VB6 projects may be getting compiled to a different location, in which case this will not behave as you expect (see below).
    2. OutputFiles not getting copied to the drop location.

    By default, the DropBuild target (corresponding to the "Copying binaries to drop location" build step) just copies the contents of the Binaries directory to the drop location.  If your VB6 code is getting compiled to some other location, the binaries will not get copied over by default.  You have a couple of options here – you can either figure out how to tell VB6 to compile your binaries into the Binaries location (you should be able to use the $(BinariesRoot) property to get this directory), or you can override the AfterDropBuild target and copy the files to the drop location yourself.

    FYI – we don’t recommend overriding the CoreXX targets in general, since these contain our default logic and overriding them is likely to break your build when you upgrade, have unintended side effects, etc.

  12. AlexeyYumashin says:

    Hi Aaron, and thanks for your answers!

    Item 1: after the line ‘m_exitCode = p.ExitCode;’ is executed, the m_exitCode variable = 1. This leads to logging my custom error message ("VB6 Compiler Error: …") and to error count increase (‘m_errors++;’). At last, the following line is executed: ‘return (p.ExitCode == 0);’ – so the ExecuteInternal function returns FALSE. In the BuildLog.txt I see my custom error message and the ‘Done executing task "VB6BuildTask" — FAILED.’ as well. Very strange, because at the same time "Build steps = Successfully completed"!

    I can send you BuildLog.txt and VS screenshot (with build output) if it makes sense. My email is a.yumashin@gmail.com.

    Item 2: you’re absolutely right!

    Item 3: haven’t checked yet… anyway I decided that default drop location will not work in my case, so I’ll use the AfterDropBuild target.

  13. Sorry for the delayed response Alexey – things have been really crazy here the last few weeks.  

    I suspect the issue here is that while MSBuild realizes that the build has failed, Team Build does not. Try something like the following and see if it helps:

    <Target Name="AfterCompile">

       <NonMSBuildProject TeamFoundationServerUrl="$(TeamFoundationServerUrl)"

                          BuildUri="$(BuildUri)"

                          …/>

       <OnError Targets="FailTheBuild;SetBuildBreakProperties;OnBuildBreak;" />

     </Target>

     <Target Name="FailTheBuild">

       <SetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)"

                           BuildUri="$(BuildUri)"

                           CompilationStatus="Failed" />

     </Target>

  14. AlexeyYumashin says:

    Hi Aaron,

    Thank you for the answer! I’m also repying with a big delay :) sometimes notifications from this site do not reach me, or maybe are mistreated by the spam filter, I don’t know exactly.

    Well, I understand the whole idea, but … what’s the SetBuildBreakProperties target? and what’s the SetBuildProperties task? I get the following errors:

    error MSB4057: The target "SetBuildBreakProperties" does not exist in the project.

    error MSB4036: The "SetBuildProperties" task was not found.

    I inspected the Microsoft.TeamFoundation.Build.targets file but found no one of those. Maybe it’s smth. from TFS 2008?! (and I’m using TFS 2005!)

  15. AlexeyYumashin says:

    It’s again me.

    I suppose you were speaking of the 2008 version.

    I used another approach:

     <Target Name="FailTheBuild">

       <CreateProperty Value="Failed">

         <Output TaskParameter="Value" PropertyName="CompilationStatus" />

       </CreateProperty>

        <Message Text=" ** CompilationStatus = $(CompilationStatus)" Importance="high" />

     </Target>

    I think it should produce the same rezult as your SetBuildProperties task.

    So… this doesn’t help! Message task shows that "CompilationStatus = Failed",  but I still have "Build steps = Successfully completed" :((

    Maybe the "CompilationStatus" property is also typical for 2008 version only?!

  16. AlexeyYumashin says:

    Well, I can solve the problem if instead of returning p.ExitCode I’ll simply throw an exception. In such case "Build Steps = Failed".

    But I think it’s not the 100% correct approach because several (but not all) build steps do occur AFTER this failure:

    • generating list of changesets,
    • copying binaries to drop location,

    • copying log files to drop location.

    I noticed that if standard (not custom) tasks fail – the whole process is stopped. Maximum – a bug will be created.

    I don’t think that "generating list of changesets" or "copying binaries to drop location" is the correct thing to do after compilation failure… at least because there are no any binaries in such case :))

  17. Alexey –

    The SetBuildProperties task is indeed new to 2008 – sorry for the confusion.  

    As for the last question – because your task is executing inside the Compile target, a failure results in the OnError targets executing just as if a compilation error had occurred.  On a compilation failure, we do indeed generate the list of changesets (so you can figure out which one broke the build), copy "binaries" to the drop location (whatever did get generated – sometimes, for example, the debug configuraiton might have been built before the release configuration failed), etc.

    If you don’t like this behavior, you could put your logic into some other target (e.g. BeforeTest) that doesn’t include the OnError handling.

    -Aaron

  18. AlexeyYumashin says:

    Aaron,

    Thank you for the feedback.

    It’s clear with the last question. But why the $(CompilationStatus) property – created (in my case) by the CreateProperty instruction – doesn’t have any effect? is $(CompilationStatus) also new to 2008 version?! should I continue struggle with ExitCode – or should I forget and use exceptions instead?

    Alexey.

  19. The key to the CompilationStatus logic is that the SetBuildProperties task actually sets properties of the IBuildDetail that represents the current build (IBuildDetail is part of the new 2008 Object Model, and doesn’t exist in 2005).  That is, IBuildDetail has a property called CompilationStatus that is part of what determines the overall status of the build upon completion.  Setting a local (MSBuild-only) property name CompilationStatus doesn’t have any impact on the build in either 2005 or 2008.

    -Aaron

  20. AlexeyYumashin says:

    Thank you. Well, I need to move to 2008 version :)

    One question more.

    Could you please take a look at my post in MSDN forums? Here it is:

    http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2968018&SiteID=1

    I got a question regarding logic of creating list of associated changesets. I’m not sure I explained everything there clear enough… but I’m ready to provide more detailed info if needed.

    Thank you in advance!

  21. BuciboLebo says:

    The problem I am having is that where there is a syntax or other type of error, VB pops up a message box instead of writing to standard error

  22. AlexeyYumashin says:

    BuciboLebo,

    it’s strange because we have there psi.RedirectStandardError = true. Anyway, there’s a workaround: in the command-line arguments (used to start vb6.exe) specify "/out:<filename>" (or "/output:<filename>" – plz run "vb6.exe /?" to see the correct syntax) – and all output will go to the file you give there.

  23. NithishBabu says:

    Where I can find a demeo of MSBuild Extension?