A new flag to optimize ASP.NET compilation behavior


Update (9/04/2009)

Actually downloading this fix for 3.5SP1 has been a confusing topic.  Here are links from which you can download it directly without having to call Product Support.  Note that even though the links appear to refer to an unrelated issue, the binaries do in fact contain this fix.  Sorry for the confusion!

For XP and Server 2003: go here

For Vista and Server 2008: go here

For Windows 7, the change is already in there, so you do not need to install any hot fix!

—————-

Quick summary: we are introducing a new optimizeCompilations switch in ASP.NET that can greatly improve the compilation speed in some scenarios.  There are some catches, so read on for more details.  This switch is currently available as a QFE for 3.5SP1, and will be part of VS 2010.

To turn on this switch, add it to your compilation section:

  <compilation optimizeCompilations="true">

 

What prompted us to add this switch

The ASP.NET compilation system takes a very conservative approach which causes it to wipe out any previous work that it has done any time a ‘top level’ file changes. ‘Top level’ files include anything in bin and App_Code, as well as global.asax. While this works fine for small apps, it becomes nearly unusable for very large apps. E.g. a customer was running into a case where it was taking 10 minutes to refresh a page after making any change to a ‘bin’ assembly.

To ease the pain, we added an ‘optimized’ compilation mode which takes a much less conservative approach to recompilation. The drawback is that there are some edge cases where it does not do the right thing, which is why it had to be an optional setting. But those cases are rare enough to still make the feature extremely useful for people running into these issues.

 

Why it is not necessary to always fully recompile

All the pages (aspx/ascx/master) in an ASP.NET site are compiled with a reference to top level files (bin assemblies, App_Code and global.asax). So it feels natural to think that if any of these change, we need to recompile everything that depends on them.

However, thinking about it a little deeper reveals that it is not the case, and that more often than not pages don’t need to be recompiled. Let’s look at the various types of changes that may occur in the top level assemblies.

Change to a method implementation

That’s the easiest case, and is clearly harmless. You’re not changing any public API signature, so any page compiled against the old version will continue to run fine without requiring recompilation.

Addition of new APIs

You add a class, or some new public members on an existing class. Since these things didn’t exist before, none of the already-compiled pages are using them, and there is no need to recompile.

Addition of a CLR attribute to an existing member

This is the typical Dynamic Data scenario where you add attributes like DisplayName to properties. Since all the CLR attributes are discovered at runtime via reflection, pages don’t need to be recompiled.

Rename or deletion of APIs

Now it starts getting interesting. You had a method Foo() that some pages were calling, and now it’s gone. Clearly, the old page assembly is no longer valid. But let’s compare what happens if you recompile the page vs. if you don’t:

  1. You recompile the page: you get a compile error because it’s using a non-existent method
  2. You don’t recompile the page: you get a MissingMethodException, and the exception message includes the name of the missing method

So in both cases you get an error, but the error in the second case is not quite as clear as in the first case. But still, it is pretty usable, so it is not a big degradation.

Change to a type used in an existing API’s signature

This is the nastiest case. E.g. suppose your Foo() method returns an ‘int’, and you change it to return a ‘short’. Now suppose you have a page that calls ‘Response.Write(o.Foo())’. Let’s compare what happens if you recompile the page vs. if you don’t

  1. You recompile the page: everything compiles and runs fine, because the same C# (or VB) code can be used to call the method, no matter what its return type is.
  2. You don’t recompile the page: you get a MissingMethodException, because the previously compiled code cannot work with the new method signature.

Undeniably, this case is broken, and is the primary reason that we can’t turn on this optimization by default. Luckily, in practice this situation in not extremely common, which is why the optimization is still very usable for users that are aware of the limitations.

 

What exactly is the optimization doing

ASP.NET uses a per application hash code which includes the state of a number of things, including the bin and App_Code folder, and global.asax. Whenever an ASP.NET app domain starts, it checks if this hash code has changed from what it previously computed. If it has, then the entire codegen folder (where compiled and shadow copied assemblies live) is wiped out.

When this optimization is turned on (via <compilation optimizeCompilations=”true” />), the hash no longer takes into account bin, App_Code and global.asax. As a result, if those change we don’t wipe out the codegen folder.

Of course, if App_Code or global.asax have changed, they get rebuilt. But pages that have been previously compiled don’t (unless they have themselves changed).

Note that this optimization doesn’t affect the app domain shutdown behavior. i.e. we still shut down the domain as soon as any top level file changes. It’s only the amount of compilation in the new domain that is affected.

Comments (30)

  1. Guy kolbis says:

    David Ebbo posted about a new flag to optimize ASP.NET compilation behavior. According to David: “we

  2. Thank you for submitting this cool story – Trackback from DotNetShoutout

  3. What's New says:

    Quick summary : we are introducing a new optimizeCompilations switch in ASP.NET that can greatly improve

  4. Cuando desarrollamos una aplicaci&oacute;n Web en ASP.NET, surge de acuerdo a algunos escenarios que

  5. You’ve been kicked (a good thing) – Trackback from DotNetKicks.com

  6. DotNetBurner – burning hot .net content

  7. I don’t think "optimizeCompilations" is a very good name for this setting.  It’s not optimizing the compilation, it’s simply doing incremental compilations rather than full compilations.

    Anybody who sees this setting will immediately think, "Why yes, I want my compilation to be optimized!" and will enable it without a second thought.  A name like "incrementalCompilations" would be much more meaningful and less confusing.

  8. davidebb says:

    James, agreed, the name is not great.  Incremental might be confusing as well, since in many cases, the compilation behavior is already incremental (e.g. when you touch an aspx page).  Naming is always the hardest part! 🙂

    David

  9. Esben Sundgaard says:

    This sounds really great! Our recompile time is more than a minute. But why do we have to contact Microsoft to obtain this fix? Why can’t we just download and install it right away 🙁

  10. davidebb says:

    Esben, I agree it would be simpler if the fix would be directly downloadable rather than having to call Customer Support to get it.  I’ll look into whether we can make this easier.

  11. Jarle says:

    In my opinon, all hotfixes should be available for download throught the help and support site. Contacting customer support is what people used the internet for in the 90’s….

  12. davidebb says:

    Jarle, I agree, things are more complicated than they should be. Note that for Vista and Server 2008, the QFE is actually available for download.  Just search for ‘Vista’ in the QFE page. The download comes with another QFE as well, which has an unrelated name, but it does in fact contain this QFE.

  13. Where can I download this fix for Windows XP?

  14. davidebb says:

    Ricardo, see my update at the beginning of this post.

  15. Osvaldo says:

    Does anybody can confirm if a create/delete a folder dynamically in the application virtual directory will cause the iis to recompile?

  16. davidebb says:

    Osvaldo: creating/deleting folders should not cause any recompilation (though it may cause the app domain to cycle).

  17. David Ebbo’s blog : A new flag to optimize ASP.NET compilation behaviorSource: blogs.msdn.comUpdate (4/22/2009) Actually downloading this fix for 3.5SP1 has been a confusing topic. Here are links from which you can download it directly without having

  18. David Ebbo’s blog : A new flag to optimize ASP.NET compilation behaviorSource: blogs.msdn.comUpdate (4/22/2009) Actually downloading this fix for 3.5SP1 has been a confusing topic. Here are links from which you can download it directly without having

  19. Jon Adams says:

    I second James Messinger’s comment about a better name for the option.

    My suggestion (note the reversed logic):

    <compilation AlwaysRecompileOnChanges="false" />

  20. Ville says:

    Tried to download for XP. After registeration I get this message:

    Page Not Found

    The content that you requested cannot be found or you do not have permission to view it.

    If you believe you have reached this page in error, click the Help link at the top of the page to report the issue and include this ID in your e-mail: 162bdff5-e999-4631-92a0-45ed562c5baa

  21. davidebb says:

    Ville: is that happening consistently?  Also, not that with Framework 4.0, this change comes built-in, so you don't need the install anything.  This is only relevant for the older framework 3.5.

  22. stohn says:

    404 for the "For XP and Server 2003" link.  It happens consistently.  The Server 2008 link works.

  23. davidebb says:

    @stohn: I fixed the XP link.

  24. mike nelson says:

    This brings up an interesting point about restarting the app every time a file changes. That as also extremely overzealous. It applies to any type of file in a number of specific folders. For example, putting a text file into the bin folder restarts the app. It would be awesome to have a "optimizeAppRestarts" flag which takes the same view on restarting the app. Basically if it didn't need to recompile, then why does it need to restart? (Restarting and downtime why files are copying is a huge pain and is often unnecessary)

  25. davidebb says:

    @Mike: it would be nice indeed, but in practice would be difficult to implement. The root of the problem is that the CLR doesn't allow assemblies to be unloaded in an AppDomain. Do if you have foo.dll in bin and it gets loaded, we can never have a different foo.dll loaded in that AppDomain. That's why we need to spin up a new AppDomain in those cases.

  26. Edge says:

    nice one. Is this applied to SharePoint as well?

  27. davidebb says:

    @Edge: as I recall, Sharepoint uses non-compiled pages, which should make this not relevant. But I'm no Sharepoint expert, so I may be wrong!

  28. Morgan says:

    Does this apply to ASP.NET Web Applications and Web Sites, or only to Web Sites?

  29. David Ebbo says:

    @Morgan: it applies to any compilation that happens at runtime. For Web Applications, the views are still compiled at runtime.