Help me, help you!


About a month ago I talked about the problems people have when trying to use Visual C# in conjunction with an external project system, as opposed to the project system built into Visual Studio.  Currently when you try to use C# files outside of a project we give you a pretty bad experience; intellisense tends to fall flat on it’s face and unless you’re using something defined in mscorlib.dll we won’t really be able to help you.  I’m pretty biased on this issue and I’d prefer if everyone used the project system, but I realize that that won’t always be the case.  There are certainly those that use any number of other excellent editors to get work done, and VS should be a tool that integrates well into whatever system you currently have working for you instead of forcing you to re-architutre your processes.


However, the issue isn’t completely simple.  Without a project we feel that (at least for VS2005) there is no way to have behavior that solves everyone’s needs.  For example, if you open up a .cs file outside of a project and try to reference a type that we don’t know about, what should we do about it? We could search through metadata for it, but what if we found the wrong type?  If you have a type Foo that references Bar and you have two files open that define Bar which do we resolve to? (that case is more common when you’re viewing different versions of a file from source control).  If you ask to build what should happen? etc. etc.


I’m bringing this up because it’s something we’re interested in improving and I’ve been thinking about ways to do it (while taking into account the time left before we ship).  One way is to simply parse and try to understand the file you have open in the context of all the other files you have open.  So if you have a file Foo.cs open that has a type Foo that references Bar, we won’t understand Bar unless you open Bar.cs.  As for references, we might use just mscorlib.dll, the set of references you have in the current solution you have open, or maybe some default list that’s defined elsewhere (and available for you to modify).  You probably wouldn’t be able to build (as I’m not sure what that would mean), but most other things would work.


Is this a scenario that you run into often enough that you would like it to be handled better?  Or is the current Visual Studio based project system working?


From the last post:


“There’s a lot of information about MSBuild out there, so I won’t repeat it.  However, there is one really nice thing about it’s integration in VS2005.  Specifically, when you do a build from within the IDE you get the same build system as when you run MSBuild externally on the project file.  So we have a system now that will allow you to build internally and run from your existing build system without any changes, and which will produce the exact same results.


So what do you think?  Is it worth adding that feature that will allow C# to work outside of a project?  In your own organizations (or on your own) do you find yourself wishing for this feature?  Or are you ok with the support we currently have.  If you’re not ok, would the new MSBuild system solve your problems, or do you need support that goes far beyond that.  Do you think that this is just an MS issue and that we should fix our internal processes rather than adding more complexity to an already complex tool?”


Feedback is much appreciated here.


Comments (15)

  1. Nicholas Allen says:

    I use Subversion with VS2003 or at least attempt to and it’s pretty excruciating. There is a halfway decent addin AnkhSVN that works well if you keep a traditional solution/project structure along with your repository. I’d really like to just point VS at my repository and have it pretend there’s something like a project there. Minimally it would need to recursively include and exclude files/directories by name. It’s ok if it can’t build directly or write project metadata as long as things like Intellisense work. I actually prototyped an addin to create a fictional project on load from a local repository copy. But I shelved it because VS extensibility is about as painful as doing things by hand and compatibility between versions of VS is not so hot.

  2. Darren Oakey says:

    For me, I’d say the project based system would be/is fine… if it worked as expected.

    Let me explain.

    At my previous contract, we had about 160 "components" – each component corresponded to one project. Here we have about 12.

    If you choose project-references, even though you structure your components in a tree, when you are editing the "main" app – you have to have ALL your projects loaded. As far as I can tell, the IDE gets exponentially slower with every project – so with 12 it’s scary… with 160 it’s impossible.

    However, if you reference by dll’s… you only seem to get one compile. You compile. Yay it all works. You do anything else… oh no! dlls left right and center are locked. Old copies of the dll are in this this and this directory. Aarrgh… it will never work again!

    We finally, after much trial and error (and ignoring/inverting almost every suggestion in the original MS whitepaper about how to structure your projects, that we originally tried to follow) – found a situation that’s workable – and that is:

    * All projects are in one solution (we ditched soln hierarchies – because that just meant you alternated between various states of horrendous, depending on what level you were coding 🙂 ).

    * All projects are by default _unloaded_

    * All projects are set to compile to a single, common BIN Directory (drastically improves compile times too btw – because you don’t have 2 million copies of common.dll on your disk, post compile :))

    * To make a change, you right click and load the project you want to change. You make the change, and, on those rare occasions you change an interface, you BY HAND guess what projects this might affect and "load" those projects and compile them

    * you are CAREFUL that you never have more than one project loaded into the IDE at one time- because if you do, and you accidentally hit compile, then dlls get loaded into memory, locked, and you have to restart VS before you can continue working.

    This works… it actually works surprisingly well if you are careful. You have access to the entire source tree, but since you only ever have one project open in the ide at one time, everything is lightning fast, even compiling and running. You can open up a base level dll, make a bug fix, and hit F5, and see your app start running. You don’t have to wait for a rebuild of the 120 projects that you know reference that dll, but you also know don’t need to be rebuilt because you haven’t changed the public interface of the DLL.

    HOWEVER

    What would be nice would be if you could use project references, "unload" a project, and have it still work – I don’t understand why it doesn’t!?! – the project references are still all pointing to the right dll – the dll is there, it works, it’s up to date?….

    Anyway… that’s my dream. Our way, annoying as it is, works for the moment – you just have to be careful.

  3. Kristof Verbiest says:

    We’ve had the same problems as Darren here at work. Our software is about the same size (~150 projects), and it is a real nightmare to use Visual Studio with this amount of projects.

    I’ll give a short overview of the problems we encountered:

    * all projects in one solution doesn’t work (too slow). We didn’t try the trick that Darren suggests, but it still seems very error-prone and I fear for the load-time when Visual Studio tries to load this huge amount of projects.

    * We also put all the binaries in a common BIN directory. This worked great, until one day when one of the binaries became bigger than 64KB. From that day on, we’ve been having the problems that Darren describes (building does not work because files are locked). We managed to find out what is causing this locking: if you reference a binary that is bigger than 64KB, it seems that Intellisense is creating a memory-mapped file for this reference (instead of just loading it into memory). This has the side-effect that this file is locked, so you cannot build it anymore.

    * Currently, we have this temporary solution: we compile every project into a separate BIN directory (this is the default in VS). Then, we copy the generated binaries to our common BIN using a post-build macro. This works great, but it is rather slow because of the additional copy that is taken.

    This has given me a lot of headaches, and I really hope that using the new MSBuild system, it is possible to develop big software projects without these kinds of problems. In the past (using Visual C++ 6.0), we have developed even bigger projects and we never had problems like that (of course, back then, we had other problems… It seems like you can’t have it all…).

  4. Nicholas: Thanks for the info. I work with a guy who’s worked on subversion, i’m wondering if he’s got a suggestion for this 🙂

  5. Darren/Kristof:

    We’ve word hard on improving the experience with large solutions with many projects in them. Especially WRT to the file locking issue. I don’t a lot of details about this, and I think we would find it very helpful if you tried out the betas and let us know if we were scaling to meet your needs.

    WRT to the work C# has done, when making changes to how intellisense works, I try to test it against a solution we have with 30 MB of source to make sure that it doesn’t screw anything up. Unfortunately, there are so many components doing things simultaneously (especially when you load a project) that you can always end up screwing things up.

    Of course, when that happens we usually get some nice emails from people complaining and we fix ASAP.

    Again, it would be incredibly helpful if you tried out the new 2k5 project system to see if it solves these problems.

    We’re aware of some issues wrt c# referencing files from c++. However, we think that for c# referencing c# it should be a lot better.

  6. Darren/Kristof:

    I can go into more detail as to how we’ve improved the situation, but here’s one big way we did it:

    When you make a C# to C# (C2C) project reference it’s now no longer the same as a VS2002/2003 C2C project reference. In 2k3 when you had this reference it wasn’t a "live" reference, but instead was just a reference to the dll built by the other project. Because of this you might have observed that changes you made in one project weren’t observable in another project until you did a recompile (and thus the other project would read in the new metadata from the first project).

    Now in 2k5 we’ve made C2C references "live". So, instead of referencing the output of that project you reference the project itself. This means that when you make a change in one project it’s immediately available in the other. It also means that because a C# project no longer references another projects dll, there are far less chances of a locking issue because, well, no one is trying to read the dll produced by another project.

    Now, I realize that sounds kind of weak as you might think that that meant that we were "solving" the problem by just trying to avoid it. That’s not the case here, lots of work was also done to make sure that we wouldn’t lock dlls when you were actually referencing.

    Note: it’s somewhat of a difficult problem. Imagine we have a reference to a dll. We copy it, release it, read all the metadata out of the copy. We also place a file system watcher on the file so that we’re notified if anything changes it. Now imagine that some tool intends to make changes to the file in two steps. On the first step they make a small change to the file, we get notified of that and we try to copy the dll, then while we’re copying it, the tool tries to make a second change and sees that we’ve locked it (even thought we’re about to unlock it a few seconds later).

    does this information help out at all?

  7. Dr Pizza says:

    "So what do you think? Is it worth adding that feature that will allow C# to work outside of a project? "

    I don’t see why. If you don’t want to use the IDE, just don’t use it.

  8. Dr Pizza says:

    "Now in 2k5 we’ve made C2C references "live"."

    Does this liveness have the smarts to work when referencing e.g. MC++ from C#?

    (IOW, something to fit the kind of development scenario I described elsewhere)

  9. DavidL says:

    When working on web serivces it would be nice to be able to edit C# source files without needing the entire solution open. Currently the IDE wants to connect to IIS and a virtual directory, and that’s a pain if you don’t have IIS or a web server on a dev machine. I think it prevents you from opening the project (it’s been a while since I’ve tried it so I don’t recall the exact error).

    We also have the same problem as Darren and Kristof – VS does not scale well to solutions with large numbers of projects.

  10. To Dr. Pizza

    If your build system is traditional makefile-based, you do not have a choice.

  11. DavidL: YOu should send that feedback to kevin blogs.msdn.com/KevinPilchBisson

    He works on the C#+Web Developer integration.

  12. Dr.Pizza: C++-to-C#: Right now no. I’d like to have a unified type model in the future so that any language can plug in and get always up to information from any other language about the types, etc, that it has.

    I think it would be excellent both for just people using the editors, and for people writing plugins to VS where they want to be able to know about all that information.

    It’s also a must to be able to support refactoring over large multi-lingual solutions in the future.

  13. Mikhail: But as part of a traditional muild system you can certainly call into msbuild which processes c# project files just fine 🙂

    I thought flexibility was a ncessary part of any build system. 🙂

  14. Dr Pizza says:

    "If your build system is traditional makefile-based, you do not have a choice. "

    You could use a dumb text editor.

    After all, you’re already using a dumb build tool, so why not go the whole hog?