Posting suggestions for the future

Several people in my department went to Teched and they came back with some general impressions of what kinds of performance problems our customers were encountering broadly.  Of course they also came back with impressions in other areas but those are clearly not as important as the performance issues so I’m ignoring those 🙂 🙂 🙂

Ahem. 🙂

Some of the themes were:

  • The “drawback” of having such a rich set of APIs is that developers just pick APIs that work without often knowing/realizing what the performance implications of using them are. It seemed like it would be great if we had some means to include this kind of information in the documentation.

  • It’s still difficult to know what to do when they have a performance problem, what tool to use etc.

  • There are not enough guidelines on how they can design their application so they can meet the performance goals or make their existing applications run faster

Not wanting to assume that my readers necessarily have the same needs as Microsoft customers generally, I was wondering which one of these points you all would be most intersted in hearing about.  Or maybe something else entirely.

I have thoughts in all those areas and more besides.  🙂

Thanks for your feedback in advance.

Comments (14)

  1. Norman Diamond says:

    > The “drawback” of having such a rich set

    > of APIs is that developers just pick APIs

    > that work without often knowing/realizing

    > what the performance implications of using

    > them are.

    For 90% of development that is really a reasonable choice.  At this level a more important question is whether the APIs work, and in some circles it’s more important for an API to work even if slowly, than for the API to not work even if quickly.  If you want the wrong answer as quickly as possible then you shouldn’t even call an API, just assign a constant.

    For the remaining 10%, things can vary.  I had loads of fun last week teaching a colleague how arrays work in C++.  The subscript didn’t have to be stored in the array together with the value of the element.  My colleague had been promoted because he had written some acceptable programs in C#, and I guess he had used dictionary style structures without ever knowing about plain old arrays.  The reason I had to teach arrays to him is that a spec had come in from outside and an array was already defined in the spec, but he didn’t understand my code.  Anyway, if an array is a suitable object then of course the difference in performance between an array and a dictionary is huge.  Nonetheless if a dictionary works fast enough for the program then maybe it’s not the place to start worrying.  Profile the code and see which part of it is taking 30% of the CPU; that’s the place to start optimizing.

  2. JConwell says:

    I’m a performance engineer for a software company that uses .Net, and the biggest performance issue that I see in .Net and the massive class library that ships with it, is that there are usually 5 or more different ways to do any given piece of functionality (pretty broad statement, I know), from fairly easy to implement to fairly complex.  And more often than not the way the .Net Framework teams have developed their API, the easier something is to use, the bigger a performance hit it is.  The average "Mort" developer out there learns one way to do a thing (usually the easiest) and sticks with it forever, and doesn’t dig around and test to see how much more efficient a different way might be.  

    As a perf engineer, I’m not usually trying to figure out more efficient memory optimizations, or looping consolidations.  Its usually re-writing how apps do their file IO or data access strategies.  Very simple things that end up giving us huge time savings.

    So my theme would be to simplify the API, but with an emphasis on performance.  Don’t give us so many types of guns to shot ourselves with.  Give use one or two options that are well thought out and efficient.  We’re developers, we’re fairly smart, don’t dumb down the API for us.

  3. Peter Ritchie says:

    The code I encounter with my clients is almost always programming by coincidence [1] cargo cult programming [2] or voodoo programming [3].  For the most part, programmer’s simply throw together code in the hopes it will work and when it finally does, it’s done.  Of course, there’s far more than performance ramifications to this; but that’s one problem.

    The biggest problem, in terms of .NET, is the documentation.  Better examples need to be added to MSDN.  Programmers search by class or method name first (what’s the point of having intelligible class/method names if they can’t be used for searching).  The area in the see also should also be expanded to include links to classes/methods that would be a better choice in the general case (and point that out in the seealso).

    It also doesn’t help much that the framework offers many different ways to do things with only one "correct" way.




  4. Ben Craig says:

    One thing that would be of enormous help to me is some kind of hint at performance characteristics of each API call.  I’d love to see something vaguely like big O notation, but that that doesn’t really apply too well to a lot of API calls.

    Perhaps something like "This function generates roughly 10^3 instructions and modifies less than 512 bytes on the hard drive".

  5. c says:

    I’d love to see more discussion of tools to use in analysis.

    A million bonus points if you provide a pointer to a tool that will make it easy to find the "expensive things" in my ASP.NET apps.

  6. Abdul Aleem says:

    I strongly agree with Peter Peter Ritchie’s examples point, MSDN is missing examples at many places in .NET API Documentation. Also, the description and Remarks sections for API’s needs to be more explanatory, and should address the Performance of the API and dependencies etc.

    From the design perspective, MSDN Patterns and Practices are doing a great Job but they still need to imporve the Patterns and Practices guides.

  7. Uri Cohen says:

    Say I have a complex managed system and I wish to improve its starting time. In a multithreaded environment this is not a trivial task at all, since the main thread is waiting on tasks performed by various different threads.

    How do I analyse ‘what is blocking my main thread and for how much time?’

  8. Laura T. says:

    I’d like to see some behavior analysis/diagnostics related postings,

    like "If you see behavior X in your apps (that is, when counter x is above y, value z is below c…) then you might check A,B,C because your app might exhbibit symptoms D".


  9. Pawel Pabich says:

    Well, performance field is not easy and I don’t see any simple way how to enable thousands to be good at it. In fact only deep knowledge can help and in my humble opinion there are not enough articles and books that talk about this topic. I’m not saying

    about short posts although they are useful and interesting. I’m saying about long MSDN articles.

  10. Specific documentation about the performance implications of the different ways of doing things (the first bullet point in your post) would be especially helpful to developers, especially those of us in a mentoring role.

    I would be very interested specifically in an authoritative response to the ongoing debate over whether or not the VB throwback functions (CStr, FileOpen) should be abandoned in favor of their native .NET counterparts strictly for performance reasons (leaving all other arguments aside).

  11. I would like to see a combination of points 2 and 3.  

    I think that the analysis tools need a usability review.  Find out what problems people are usually trying to diagnose and build tools around those scenarios to make the analysis easier and more approachable.

    Along the same lines, there needs to be more guidance about how to build applications with performance in mind.  More to the point, since it’s quite obvious that most people don’t build it correctly the first time around, some literature that talks about how to fix common performance issues in an existing system would be great.

  12. JamesKovacs says:

    I agree with the other commenters who mentioned better documentation in the MSDN regarding the order-of-magnitude cost of methods. This would definitely help.

    I would have to disagree with the commenters asking for the BCL to be simplified. In the vast majority of scenarios, the simple APIs with mediocre perf are sufficient. This makes the entire platform more approachable. Even if the simplest overload for a certain file I/O operation gives poor performance for multi-gigabyte files, it just doesn’t matter if you’re writing an application that edits small text files. Many people also complain about System.Xml. If you’re writing an application that requires fast start-up, don’t use System.Xml to parse your config or keep your config in non-XML format. For many applications, the additional overhead of System.Xml is negligible compared to other startup costs such as querying a database for a list of customers. So why make most developers’ lives harder because some developers need really fast app start-up?

    One big hurdle I see in educating developers regarding performance is the difficulty in collecting good perf statistics. Which tools to use? When to use them? How to interpret the data? Yes, this blog and others provide good guidance, but it takes a long time to understand instrumenting profilers, sampling profilers, perfmon, SqlProfiler, ClrProfiler, … Compounding this is the fact that there are so many ways to collect bad or meaningless statistics. There is a steep learning curve for understanding application performance. I don’t know an easy solution to any of these issues except developer education.

  13. Alois Kraus says:

    Hi Rico,

    I think the .NET API’s are feature rich enough to suit most needs, including performance. If the "normal" API is too slow I can still write my own but I hope that will not be too often. In big projects the main perf problem is that nobody cares (guidelines or not) except for some perf engineers that are responsible. I think that is too late in the game if everybody codes as he wants and hopes that his inefficient code is not noticed until some acceptance criteria is not met but no time is left to fix it. There are many good tools out there I must admit that most of them are too complicated to for daily use. We need simple tools which can be integrated with unit tests so every developer is enabled to code perf aware and does get better understanding what things really costs.


     Alois Kraus

  14. Mark Sowul says:

    I just think it’s a pity that only Team Studio gets the built in VS performance profiling tools (and the integrated fxcop, and unit testing).  I don’t think these features are only needed by teams.