Experts and non experts


I had an interesting conversation with a colleague of mine last week. We were discussing
the extent to which full details of a technology or feature should be exposed to end-users.
We were discussing this issue in the context of the IDE specifically, and how details
should be exposed in the UI, but the issue is also relevant when considering APIs.

For example, consider the System.Xml namespace. This namespace exposes a set of types
that provide standards-based support for processing XML. For example, the XmlDocument
class implements the W3C Document Object Model (DOM) Level 1 Core and the Core DOM
Level 2.

XmlDocument exposes members such as CreateDocumentFragment, CreateElement and CreateNode.
If you’re familiar with the details of the W3C DOM you’ll know which method to choose
when you want to add a new data record to an Xml document that you’re using as a database.
But if you’re not as familiar with the W3C DOM, you may not know which method to use
to accomplish your task. You’d need to become familiar with the DOM before fully understanding
what the different methods do and when you should use which one. Likewise, if you
wanted to run a query over your data, you’d need to know about XPath and that you
create an XPath expression to select the nodes in your document that meet the search
criteria.

This is all fine if you are comfortable and familiar with all things XML. But if you’re
not, it may be somewhat daunting - there is a lot to learn. The rewards are potentially
pretty large, but the tax you have to pay to reap those rewards may also be pretty
large.

The question is, does the developer always have to be forced to pay this much
tax or is there a way that the API can be designed such that the developer who does
not have as much familiarity with all things Xml can still reap the benefits?

It’s basically a trade-off between making experts comfortable and providing them with
an API that maps very well on to their expert view of the domain at the expense of
requiring non domain experts to become domain experts or to forever remain uncomfortable
using the API. Or, making non-domain experts comfortable with an API by abstracting
over some of the domain specific aspects of the API but by so doing making experts
less comfortable since everything that they know about the domain will not be represented
in the API as they expect it to be.

My colleague and I concluded that it really comes down to the percentage of domain
experts and non-domain experts in the user population. We figured that if 10% of users
were experts and 90% were non experts, it would be unreasonable to force the non-experts
to become experts before they could use the API effectively. But we did not come up
with a point at which we felt it would be reasonable to make users learn a bit more
before they could use an API. Is it 50/50, or higher or lower? One argument suggests
that over time people will become experts in the domain the more that they use the
API. But another argument suggests that if the cost of being able to use an API is
too high, people won’t even bother with it, or may use the API incorrectly, leading
to inefficient code etc.

Given these arguments, at what point do you think it is reasonable to have to invest
significant time to learn about the domain that an API abstracts over? We’d love to
hear your opinion on this.

Comments (9)

  1. David Williams says:

    While I think that the System.XML namespace is a good example… a better one would System.Security. I know for my self that diving into this is needed, however, the tax, as you so aply put it, is making it very hard to get into. This is a space where access to the API is fundemental to success, but the knowledge that appears to be needed to use it correctly is far beyond the casual user. To date – I have to admit – it is beyond me. So how to handle this kind of issue?

    I personally think that it comes down to this. How vital is the API? Is it something that being a domain expert is a good thing to be and/or not to bad to learn – then the decision is not too hard. Code for the experts – and those that want to use it will come. But for something that is a day in, day out subject matter, and a very hard matter to work with – write the API to the lowest possible level of knowledge. The experts will dig a little deeper, but more programmers will at least use the API.

    David

  2. Roger says:

    Having invested a fortune in college degrees, certifications, and my own hard knocks, I believe that providing API’s to the general masses is a mistake. Programming should be left to those who understand what they are doing.

    An example was the boom in web programmers when any one with Word thought they could create web pages. I’ve spent the last several years cleaning up after programmers who thought they knew what they were doing with ASP but could not get their web application to scale.

    I think the .NET documentation could be better in several areas with better "real world based" examples. But please don’t "dumb down" the API. If you’re going to use the functionality provided by the API, the programmer should understand how to use the API.

  3. Blake says:

    Do both. Design the expert UI first, as presumably your API designers are top-flight domain experts, and then design a ‘helper’ API and implement it in terms of the expert API. Ensure that non-domain experts are involved early and often in the design of that second API. (Should that be domain non-experts? 🙂

    Put the helper API in a consistently named child namespace of the expert API, so System.Xml and System.Xml.Helper, and be disciplined about ensuring the helper API is implemented exclusively in terms of the expert API. For bonus points, ship the source for the helper APIs as a way to assist those non-experts in becoming experts if they wish.

    (Note: ‘Helper’ is probably a lousy name, but the idea remains sound.)

  4. I absolutely concur with Blake (and was just about to write something similar, until I saw he had got there first).

    As is often said in the Perl community: "There’s more than one way to do things".

  5. Nicole Calinoiu says:

    I was also about to about write something very similar to Blake’s posting. If the API is "crippled" to bring it down to the non-expert user level, experts are either going to avoid using it entirely or have to jump through some potentially nasty hoops in order to gain the lower-level control they want/need. Keeping the expert-level complexity while adding a facade that covers a reasonable set of typical non-expert usage scenarios would keep everyone much happier.

  6. Frank Hileman says:

    I hate it when people say there is a conflict between an easy to use API and a powerful API. A powerful, well-designed API should be both. If it is complex in features, but not easy to use, then the design stopped in mid process. In fact, it is much easier to design a complex, hard to use API, than an easy to use API.

    In the end, the two design goals are reinforcing, not conflicting: a complex API is not powerful if it is not easy to use. These days we have features like function overloading, and object properties, that make it possible to have both power and ease of use, without conflict, except at the level of documentation. There is no reason to say the goals are conflicting. They are not.

    In the case of the W3C XML specifications: these were not designed for ease of use, a committee designed them, so there is no point in trying to change the API that fits these specs. It is probably difficult to graft ease of use onto something like that. Instead, a new API must be designed from the ground up, providing both the power and the ease of use.

    How can an API be "crippled" by making it easier to use? "Easy to use" does not mean few properties, or few parameters to functions. The first problem can be addressed by documentation (advanced properties?) and samples; the second problem with function overloading. Where is the conflict? The number of classes? Again, this can be addressed by documentation, and namespace organization.

    I think it is a cop-out to say an API is hard to use because it is powerful. It just shows that the design process was never finished.

    To address the argument that only experts should be programmers: if anything, the trend is moving in the other direction. People have less and less time to become an "expert" in anything, and the proportion of "experts" is steadily dropping. We cannot fight this trend by designing hard to use APIs. The goal is not to change society. None of us are experts when first using an API.

  7. Philip Rieck says:

    I think I like Blake and Nicole’s answers. The other issue is learning the API: If I’m a domain expert in XML, my knowledge comes through right away. If I’m a beginner, I must learn XML.

    However, disconnect the API from the specs… now both the beginner and the expert must learn the API. The expert must allso keep the two concepts in his head separate – no one wins.

    If you make an easy to use API, and someone knows what they want to do (create a document fragment or node), why make it hard on them to slightly lower the barrier to the beginner (who will need to learn XML eventually when the magic behind the scenes isn’t working perfectly)?

  8. Anand says:

    Well as most of the others say both is the best option. But one thing to keep in mind is that the object model should be the expert one, but the helper API makes it easier to access/work with the expert object model. So over time if the user understads the system well enough to be an expert, he can move over easily by just changing the API set.

  9. Diego Vega says:

    It could be that Blake is talking about what Microsoft is trying to accomplish with Application Blocks. They are even publishing the source code to ABs so the get the bonus points!

    I absolutely think AB are the right way to address that 80% of cases when coarse grained code will cut it, also that 80% of programmers that are not experts, and even that 80% of the times when expert programmers have better things to do with their resources.

    So my wish in this regard is well designed and well documented APIs, and more and better Application Blocks.