Attributes and API usability (again!)


I’ve been running another usability study on an API that makes heavy use of attributes and have made similar observations to a previous study (see http://weblogs.asp.net/stevencl/archive/2004/05/12/130826.aspx#FeedBack)

This time we’ve been looking at the Indigo APIs. Indigo makes heavy use of attributes throughout. For example, attributes are used to identify an interface as representing a service contract or a method inside that interface as a service operation.

In this study, we’ve observed similar issues to those observed before. One of the issues that has resurfaced is the apparent reluctance of participants to consider the attributes sprinkled throughout their code as potential causes of unexpected application behavior. As in the previous study, we’ve seen developers consistently neglecting to consider modifying any of the attributes that they have applied to their code when attempting to modify the behavior of their code. In many cases, the desired effect (changing the lifetime of a service for example) is done by simply changing a property of a particular attribute. But instead, many participants spend time investigating their own code and even changing their own code to try to achieve the change in behavior they are looking for. It seems too that this behavior isn’t necessarily isolated to the usability labs. One of the Indigo team members told me of a recent example where some folks from the product team were attempting to debug some customer code. They spent a lot of time checking the IIS configuration, the customer code etc and overlooked the attributes in the code. It turned out that the wrong attribute had been used.

I think one factor in this is the low visibility of attributes. For example, one participant in the study this week was stepping through his code in the debugger when he noticed some unexpected behavior at some point during the excecution of his code. He was focused on a particular block of code and concentrated his efforts on understanding how that block of code might have caused the behavior he had just observed. The cause for that behavior was due to an attribute that had been applied to the class that defined the method the participant was stepping through. Thus when he was reading his code, the attribute was well out of his focus of attention.

However, such issues are typically an issue for the first few times only – once you learn the effect of an attribute, it’s more likely that you’ll remember that attribute and will revisit it if the same issue occurs again. And indeed, this is what we observed during the study.

Another factor involved is when there are relationships between two or more attributes. The problem gets worse when the change in application behavior necessitates modifying more than one attribute at a time. The problem is that the relationship between multiple attributes is often unclear. One of the tasks in the study I am running this week requires that participants apply an attribute to a method inside an interface declaration and another attribute to the class that implements that interface. Participants who have worked on this task have been unaware of this requirement and have not been able to complete the task without significant help. There is nothing about the design of the attributes that makes this relationship clear.

Once participants have been told about the multiple attributes and the requirement to modify them all in conjunction their reaction is always the same – “that’s kludgy!”. Having seen this in multiple studies on different APIs, I’m pretty much at the point where I would strongly recommend API teams avoid designing APIs that require developers to modify multiple attributes to achieve specific functionality. Instead, I’d recommend examining ways that the same functionality could be achieved through the use of one attribute or at the very least, consider naming the multiple attributes in such a way that the relationship between the attributes is much clearer.

It would be interesting to hear anyone’s feedback on this issue. Is this something you have experienced yourself when using an attribute heavy API? How big of an issue do you think it is? What would be your preferred solution?

Comments (14)

  1. Raph Cohn says:

    Very interesting stuff.

    With 2 plus years under our belts, we’re now cautious users of attributes. In the main, we prefer either a meta data map of some sort if their multiple elements of data to apply, or an AOP-like approach when the same behaviour needs to be applied to code with the same cross-cutting concerns. This cuts down on the visibility problem, as, with both, the metadata is defined in one place and one place only.

  2. Mike Dunn says:

    I feel that attributes are too much of a black box and I simply don’t trust them. Attributed C++ in VC 7 is IMO a dismal failure, and my experience trying to use attributes there probably colors my view of them in .Net. I much prefer the ATL way of using mix-in-style classes. For example, you can add "public ISomethingImpl<>" to your inheritance list, add a macro so the class responds to QI for ISomething, and It Just Works. All the changes are very visible and it’s easy to see if you goofed.

    My concerns stem from the fact that attributes are these magic bits you stick in certain places to effect (somtimes great) change. I tend to dislike such constructs because I can’t see the generated code, nor can I debug it, so I can’t tell if a bug is in my own code, or caused by a misused/missing attribute. For instance, I clearly remember a Avalon demo done by Box/Anderson, where their app didn’t have [STAThread] on Main() and they were stumped as to why the UI wasn’t showing properly. When Don threw [STAThread] in, it worked. I cringe at stuff like that, and to me it comes off as bad design.

    I’m also spoiled by having worked in MFC, ATL, and WTL for so long and I’ve gotten used to stepping thru those libraries’ code when necessary. With an attribute problem, that is impossible to do.

  3. Jeff Atwood says:

    I think it’s awesome that you guys are studying API usability. Programmers are terrible at usability, and guess who designs most APIs? 😉

    If you ever want a good laugh (and a case study in the above) check out the PHP API. It’s unintentionally hilarious.

    As for attributes: I agree, their use needs to be strictly limited– it’s really a special purpose tool, not a general one. The lack of locality problem is tremendous: you have an exception in code block (x) but the cause is an attribute way, way up in the function declaration.

  4. Because attributes are just data, its very hard to determine how an attribute is going to affect the behavior of a class. On its own, it does nothing.

    For me, some way of making explicit the relationship between attribute and the subsystems that act on that attribute, would be very helpfull.

    All-in-all, attributes are usefull, but far too unstructured for my liking.

  5. Steve Swartz says:

    I’m particularly interested in what people think of as alternatives. Here is a snippet of Indigo code that illustrates all of the attributes needed in Steven’s study:

    [ServiceContract]

    public class Conversation

    {

    [ServiceOperation]

    public string Greeting (string Hello);

    public string Parting (string Goodbye);

    }

    Conversation is exposed as a service. Greeting() is part of the service contract; Parting() is not.

    There are families of alternatives to these attributes that move away from a class-oriented approach. You could imagine, say, a contract container to which you add delegates keyed by their action. Microsoft has a long history of products (DCOM, Enterprise Services, .NET Remoting, ASP.NET) where you implement a service by implementing methods on a class you then expose as a service. We would like to bring these customers forward. Things like contract containers have seemed complicated to us.

    Two alternatives to [ServiceContract] are a required base class or an empty marker interface. Our experience with .NET Remoting argues that required bases classes are not acceptable to users (suppose you want to put a service contract on a WinForm). Our experience with Indigo says that people are confused by empty interfaces. Steven’s experiments seem to indicate that people are confused by interfaces altogether.

    I cannot think of an alternative to [ServiceOperation]. For the sake of consistency and security, we don’t want to light up all methods on a service just because it’s a service. We’re in a "private by default" world.

  6. Actually, the attributes that caused the most problems were those used in handling and reporting errors from the service to the client, not those used to mark a method as being part of the contract.

    The [ErrorHandling] attribute defines how a service handles errors. The [ErrorReporting] attribute defines how errors are reported to the client. Lastly, the [KnownFault] attribute identifes the exceptions that a particular [ServiceOperation] can throw.

    The values of each of these attributes has an effect on the behavior of the other attributes, to the extent that a table needs to be drawn in the documentation to demonstrate the effect.

    Two of the attributes are set on the service implementation, while the [KnownFault] attribute is set either on the [ServiceOperation] or the [ServiceContract]. The relationships between these three attributes were hard for participants to discover and to control.

    The issues above with [ServiceOperation] were largely that many participants forgot to add this attribute to an interface that had been marked as [ServiceContract] but this is one of those problems that seems to be a one time only hit.

  7. Steve Swartz says:

    I think I agree with you that [ErrorHandling] and [ErrorReporting] are bad. I think they’re too complex. I think the interdependence between them is unacceptable. They’re probably trying to do more than attributes should be asked to do.

    [KnownFault] is a different beast. In WSDL, you can assign faults to INOUT operations. We use the attribute to do this. If you want to expose the fault on all operations on a contract, you putit to the contract; if you want to expose the fault on a particular attribute, you put it on the operation. We need to be able to discover this operation/fault linkage when we’re reflecting on the type in order to auto-generate the contract metadata, and of course at runtime when we catch exceptions and generate faults.

    What are alternative ways of associating fault types with operations?

  8. Mike Dunn says:

    Mr. Swartz hit it right on the head – all the alternatives are unacceptable for one reason or another. I think multiple inheritance isn’t allowed in the CLR (right? please CMIIW, I don’t do any managed coding) so that’s out as well. I bring it up because his example code has that "ugh, hackish" feeling to it precisely because it looks like a workaround to compensate for the lack of MI. If you’re in that situation, the design (either of the API or the CLR itself) is the problem. Anything you do to simulate MI is going to look bad to _somebody_.

    I’m not sure what .Net Remoting is, but my preference as a C++ dev is to have an abstract base class and let the compiler deal with which methods need to be implemented. In that case, there’s no doubt when you leave out a method, because your code won’t compile. (This may be impossible in the CLR, but there you go. I’m at my limit of CLR knowledge here so unfortunately I can’t say much more without comparing everything to C++)

  9. Perhaps if attributes were specified as [System(attribute, attribute, …)] it might be clearer what attributes are associated with what system, and my varying the constructor, you could specify ‘schema’ of what attributes are applicable.

    Steve Schwartz, when you put a [ServiceContract] on a class, you are saying that each method of that class can optionally have a [ServiceOperation] attribute put on it, but that ‘schema’ is expressed exactly nowhere. If there was a way to specify the relationship between [ServiceContract] and [ServiceOperation] wrt the relationship between class and method, then intellisense could display a list of attribute options for each method.

  10. How is the use of attributes in Steve Swartz’ scenario different from cargo cult programming? The Indigo team seems to want developers to ignore the "man behind the curtain". Thus, Steve presents us with a couple of completely false alternatives to the use of attributes. The correct answer is pretty obvious from Steven Clarke’s analysis. The code that connects your application to the wire is no minor detail. It is an essential facet of a distributed application’s design. It shouldn’t be hidden behind an impenetrable wall of attributes and codegen.

Skip to main content