Why Can’t I Access A Protected Member From A Derived Class, Part Three


Holy goodness, I’ve been busy. The MVP Summit was fabulous for us; thanks to all who attended and gave us great feedback on our ideas for evolving the languages and tools. And I’ve been doing some longer-lead thinking and working on language futures, which I will blog about at a later date.

Last time I promised another oddity of “protected” semantics. An issue that I get asked about all the time involves the meaning of this code:

namespace Foo {
  public class Base {
    protected internal void M() { … }
  }
}

Many people believe that this means ” M is accessible to all derived classes that are in this assembly.” It does not. It actually means “M is accessible to all derived classes and to all classes in this assembly”.  That is, it is the less restrictive combination, not the more restrictive combination.

This is counterintuitive to a lot of people. I have been trying to figure out why, and I think I’ve got it. I think people conceive of internal, protected and private as restrictions from the “natural” state of public. With that model, protected internal means “apply both the protected restriction and the internal restriction”.

That’s the wrong way to think about it. Rather, internal, protected and public are weakenings of the “natural” state of private. private is the default in C#; if you want to make something have broader accessibility, you’ve got to say so. With that model, then it becomes clear that protected internal is a weaker restriction than either alone.

(As an aside: An irksome fact about the design of C# is that we do not consistently default to the more restricted mode. I like that private is the default visibility for most members and “instance”, not “virtual” is the default method behaviour. But why aren’t classes sealed by default? This would emphasize that participation in an inheritance hierarchy needs to be part of the design of a class.)

We get the feature request fairly frequently to add the “more restricted” form, but the thing is, I’m not sure what use it actually is. If the member is already marked as internal then the only people who can use it are your coworkers. What is the harm in allowing them to party on an internal member from outside the class hierarchy?

Of course, if we did add the feature, the codegen would be trivial; this kind of accessibility is already supported natively in the CLR. The work would come in defining a sensible syntax for it. protected with internal or protected and internal might work. Or, we could define a new keyword having this meaning. proternal, for example. Or intected. (The former sounds very positive; the latter, like bad news from a dentist.)

Long story short, I would not expect this feature any time soon.

Next time, some thoughts on your comments to my last entry.

 

Comments (32)

  1. oldnewthing says:

    Probably because most qualifiers combine as "more restrictive" – When you write "readonly volatile" you expect that attempts to write will fail – not that "Well, yeah, but the volatile rules don’t forbid writing, and those are the rules I’m choosing to follow instead of the readonly rules."

    Also because the word "internal" carries the semantics of a restriction – if the keywords were called "derivedvisible" and "filevisible" then maybe it would be seen more as a relaxation. "It’s derivedvisible and fileviisible, whichever works for you. It’s a floor wax and a dessert topping."

  2. Aaron says:

    Proternal either sounds like the name of a dotcom-era web company or a term for somebody in favour of same-sex adoptions.

    I think you should borrow from the accounting field and go with "dependent".  That’s really what this is – a child living in the same household.  No in-laws or great-great grandkids whom you’ve never met allowed.

  3. MKane91301 says:

    With all the admiration I have for the design of C#, I think this is one that Anders & Co. got totally wrong.

    First of all, why should I trust my coworkers? As a policy, I don’t even trust myself! If I make something as restrictive as possible, then if later I want to open it up, I actually have to think about why I restricted it in the first place. I really don’t want to be swapping bodily fluids with every class in the same assembly just because we always compile together.

    Also, if you use protected internal and virtual together, you’ve effectively made it public, because you can’t override without exposing it to every class in the same assembly with the derived class. When I have a virtual member that needs to be accessible within my assembly, I have to make two members, one protected virtual and one internal.

    I find I’m constantly in the situation in which I do not want a member visible outside of my assembly and the only classes inside my assembly that have any business with the member are derived classes. So I’m stuck with either being less restrictive than necessary or, if I can, I make it private and nest all my derived classes.

    It’s rare that I need protected or internal. In those rare cases, I would have been happy to create one protected and one internal member, which is a very easy way to create the less restrictive case when the language only supports the more restrictive case. When the language only supports the less restrictive case, it’s very hard to create the more restrictive case.

    My suggestion would be to make an attribute that could be put on protected members to make them protected and internal or one that could be put on internal members to make them protected and internal, whichever one you guys think would be more usable. Making it an attribute instead of a language feature would be easier for everybody.

  4. James Hart says:

    I’ve heard people ask why there isn’t such a combination before as well, but I don’t get it. I can’t really imagine a situation where there’s any point in allowing code in other assemblies to derive from a class, but to restrict the API accessible to external code, compared to the one you expose to internal derived classes. You’re asking to create two tiers of subclasses – those outside the baseclass’s assembly that have a restricted view of their parent, and those inside that can access more of their parent’s features. If these members are useful to internal subclasses, why aren’t they useful to external ones too? If it’s possible to develop fully functional subclasses of your baseclass without accessing your ‘proternal’ members, then why do your subclasses need them? If it’s not possible, then why are you allowing classes outside your assembly to derive from your baseclass at all?

    What people who ask for this restriction probably really want is to restrict deriving from their class to their own assembly, preventing external code from subclassing, which can be done easily enough with an internal constructor. Then any protected members are automatically also restricted to classes in your own assembly too – protected AND internal, just like they asked.

  5. EricLippert says:

    There are a number of places in C# where we use an attribute to connote some language feature — "out" vs "ref", for example, or consuming extension methods.

    That said, first, we are reluctant to add more of the same, and second, this would be a really weird one. The attribute "protected and internal" is a built-in attribute, first class in the metadata. Surfacing that with the syntax of a user-defined attribute would be weird.

  6. msbuild says:

    I heartily agree with MKane. I have learnt from experience that it is good and robust practice to expose as little as possible eve within my assembly. It is nothing to do with trusting coworkers. It is about making code easier to maintain and improve and resistant to bugs. Even my own bugs working elsewhere in the code. One might as well argue that all private methods can be internal as I trust my coworkers. Also, it is even more important to not expose external interface without conscious reason.

    Yet due to C# not exposing this modifier I am forced to use internal — to prevent exposing arbitrary external interface with all its doc, useability, versioning, compat burden — when I really just want protected semantics. Now 300 other classes have to see this member.

    The word you use to expose this is a trivial issue. Call it "protectedinternal" (all one word) if you like. Call it or "FamANDAssem" like ILDASM does. Call it zachary.

    Danmose

  7. Matt says:

    Perhaps a reminder to oneself and ones coworkers that would be sufficient would be an attribute which stopped to stop intellisense showing it outside the assembly?

    This would likely have the desired affect…

  8. kevinowen says:

    An attribute to stop intellisense from showing it already exists: System.ComponentModel.EditorBrowsableAttribute(EditorBrowsableState.Never)

  9. RichB says:

    It’s counter-intuitive because modifiers on members are more restrictive. The most obvious example is:

    private readonly int x;

    It’s not private OR readonly. It’s private AND readonly.

  10. matthieu.mezil@live.fr says:

    In MSIL, we have internal or protected and internal and protected:

    Private, FamANDAssem, Assembly (internal), Family (protected), FamORAssem (protected internal) or Public

  11. Tanveer Badar says:

    "But why aren’t classes sealed by default?"

    Would you need to think before hand if someone in future will need to derive from it or not?

  12. Jon Skeet says:

    Tanveer: You *should* think before hand whether or not someone will need to derive from it. If they will, you need to think about what implementation decisions to document. For instance, every time your implementation calls a virtual method, that needs to be documented – otherwise an override of the virtual method might decide to call the original method, leading to a stack overflow.

    Inheritance is very useful, but only where designed. In order to use it robustly, you’re pretty much forced to leak implementation details, which also limits future refactoring.

    To cut a long story short, I’m very much with Eric on this one.

    (I’m on the fence as to whether it’s appropriate to have defaults for various things at all. What if there were no default access modifiers – if you *had* to declare the access for every member and type?)

  13. David Nelson says:

    @Eric, re:"What is the harm in allowing them to party on an internal member from outside the class hierarchy?"

    Are you serious? Have you not been getting enough sleep lately? msbuild is spot on: if that were a valid argument, why even have private members, why not make everything internal? "I really don’t want to be swapping bodily fluids with every class in the same assembly just because we always compile together." I can’t imagine a better way to describe it.

    I think your analysis as to why the meaning of protected internal is counterintuitive to most people is unsound. Think of it this way. Private may be the default state for all members, but accessibility modifiers came into being specifically to enable the object-oriented concept of data hiding, or restricting access to data, as opposed to old C and other functional code where any non-local (i.e. global) variable was fair game to anyone who wanted to use it. It is therefore not surprising that most people tend to think of accessibility modifiers as restrictive rather than relaxing.

    Add to that the fact that people generally think of keywords as adding restrictions from the natural state, because in C# they generally do. abstract must be overridden, sealed can’t be inherited, const must be initialized inline, readonly can’t be written outside the constructor, fixed can’t be compacted, out parameters must be initialized prior to returning from the method, volatile can’t be optimized; virtual is pretty much the only keyword other than the accessibility modifiers which follows the relaxing rather than the restricting pattern.

    Regardless, its hard to argue that "protected or internal" is more useful than "protected and internal". I cannot remember a single instance in implement an object hierarchy when I have used protected or internal; but I can remember numerous instances when I could have used protected and internal.

    Why do you not expect this feature any time soon? Just because it will be hard to come up with a keyword?

  14. EricLippert says:

    > Have you not been getting enough sleep lately?

    Yes, I have. Have you? or do you always respond to trivial differences of opinion about technical matters with sarcastic, cutting and insulting remarks?

    > if that were a valid argument, why even have private members, why not make everything internal?

    And now you’re making a straw man attack by taking my remark out of context.

    Let me be more clear, since apparently I didn’t get my point across. If the member is intended to be private, of course it should be private. If it is intended to be accessible to internal subclasses, then what is the harm of it being accessible to internal non-subclasses?  You get NO additional security by going to "protected and internal" because your coworker can simply write a subclass to party on the member. The only benefit you get is the enforcement of your desire to have this particular internal member restricted to use from a subclass — it becomes a form of documentation.  I am in favour of self-documenting code, but in this case the benefit seems very small.

    None of that implies that we ought to throw "private" to the winds. And I have a hard time imaginging that the number of scenarios in which someone is designing a class that can be inherited one way by external code and another way by internal code is anything but a small number.

    > I think your analysis as to why the meaning of protected internal is counterintuitive to most people is unsound.

    I find it odd that you say that my argument is unsound, and then go on to restate my argument. My argument is that people find this counterintuitive because their incorrect intuition is that these keywords are restrictions, not relaxations. You say that’s unsound, and then give a whole lot of perfectly good supporting analysis in favour of my argument.

    Any time someone tells me I’m wrong and then lists of a whole bunch of points supporting my position, I get very confused. Are you sure you think my argument is unsound? Perhaps I simply did not communicate it clearly enough.

    > Why do you not expect this feature any time soon? Just because it will be hard to come up with a keyword?

    No, because we have many, many, many other far higher priorities and limited staff and time in which to deliver on those priorities. Time spent doing trivial features that benefit a tiny number of scenarios is time taken away from more important features that deliver higher value to more customers.

  15. EricLippert says:

    > What if there were no default access modifiers

    I started designing such a language once, just for fun. Didn’t get very far. I was going to call it "Verbose". You’d end up with declarations like:

    field foo: public instance nonvolatile readable writable System.Int32;

  16. Matt says:

    kevinowen : I think you missed the part where I said "outside the assembly" adding another named attribute to it that details this exceptional visibility is just fine e.g.

    UnlessWithin.Assembly | UnlessWithin.SubClasses

    It’s rather harder to implement the sub classes one since working out whether you are writing code within that context is surprisingly hard (inner classes for example) though removing from the list of possibilities suggested when intellisense sees override is pretty easy.

    The assembly one is much easier since the context of which file -> which assembly is sufficiently well defined and 1-1 that dealing with it shouldn’t be too hard.

    I’m with Eric though, much as the occasional mental disconnect on it annoys me it is so not the end of the world compared to plenty of other new languages features like dynamic blocks.

  17. Jon Skeet says:

    Eric> The verbose language…

    It depends on what you understand by "access modifiers" – I was only including the private/public/etc. Apologies if readonly, sealed, virtual etc count in terms of access – I didn’t check my use of vocabulary :(

    However, I’d also like to include sealed/virtual for type declarations as well. ("Virtual" doesn’t actually feel right – unsealed would perhaps be better.)

    In other words, for people who usually seal classes anyway, and explicitly declare private members and internal types, there’d be little change.

  18. EricLippert says:

    No, Jon, you are right — the others are not access modifiers. My point was that I was going to make a language where you had to specify EVERYTHING, not just access. It would not have been a useful language, but it would have been unambiguous. :-)

  19. int19h says:

    > But why aren’t classes sealed by default? This would emphasize that participation in an inheritance hierarchy needs to be part of the design of a class.

    There’s nothing to be gained from sealed by default, since you can’t break someone else’s code by deriving from his class. This is quite different from the case for non-virtual by default.

  20. Mike Dunn says:

    Re: classes not sealed by default. There was a video a few years ago with some senior person at MS, it might have been Don Box but I don’t remember for sure. He addressed this very issue. He also thought that ideally, classes should be sealed by default. But doing so would have created a big stumbling block fpr developers coming from C++, Java, etc. Remember, .Net was new at the time and MS wanted to make it as easy as possible for devs to start using C#.

  21. EricLippert says:

    > There’s nothing to be gained from sealed by default, since you can’t break someone else’s code by deriving from his class.

    You most certainly can! Extending a security-sensitive class to make a hostile subtype and then tricking some consumer into thinking that your object may be used safely as an instance of the benign base type is a common technique for subverting a system with poor controls on inheritance.

  22. David Nelson says:

    > Have you not been getting enough sleep lately?

    >> Yes, I have. Have you?

    Not by a long shot :)

    >> or do you always respond to trivial differences of opinion about technical matters with sarcastic, cutting and insulting remarks?

    No, not generally. But there is something about your blog that brings the sarcasm out in me. I honestly didn’t mean to insult anyone though.

    >> You get NO additional security by going to "protected and internal" because your coworker can simply write a subclass to party on the member.

    If you are referring to code access security in partially trusted environments, then you are correct, there is no benefit. However, that is not really the point; obviously we are not talking about code access security within an assembly. And you are assuming that arbitrary subclasses can be injected into the object graph at any point. In most well-designed class hierarchies this is not the case.

    >> The only benefit you get is the enforcement of your desire to have this particular internal member restricted to use from a subclass — it becomes a form of documentation.

    Essentially, you are correct. We do not have the luxury of perfect project documentation or perfect cross-team communication (oh how I wish…). The best way we have of communicating the intent of our code is the structure of the code itself. A well-designed class hierarchy screams out exactly how it should be used. I believe Krzysztof has talked about this many times. Restricting members to their minimum required accessibility and no more is the most direct way possible of indicating the desired usage of a member. I don’t want to restrict a member to "protected and internal" to STOP my coworker from using it; I want to do it to indicate to him that he shouldn’t, even when I’m not around to tell him so explicitly.

    > Why do you not expect this feature any time soon? Just because it will be hard to come up with a keyword?

    >> No, because we have many, many, many other far higher priorities…

    Good, I totally agree that this is a minor issue which should take a backseat to the many more important features and bug fixes which the community has been requesting. I just wasn’t clear whether you were saying that you thought it *shouldn’t* be done, or that it *wouldn’t* be done.

  23. Kor Nielsen says:

    James: The main problem with using a protected method with an internal constructor is that sometimes you want to use internal types as parameters to your protected method. If your class is public, that is not a possibility. So far, the best approach I’ve found is to make the class internal and expose a public interface.

  24. Josh Berke says:

    I was one of the peolpe who had always thought that protected internal would work to make things more restrictive. I never sat back and thought about how private is default so it would as you would say loosen restrictions.

    I disagree with your thought that there is no need for a "Proternal" since your trying to protect your own co-workers. One case where this comes into play is when a project enters maintneance mode and the original developers are no longer there. Something marked as "Proternal" helps to clearify the intent that only derived classes within this assembly should be using this method.

    On the same token I’d rather see other cooler language enhancements then this feature which would only be applicable a small percentage of time.

    I agree on sealing but I like that classes are not sealed by default. If developers had to consiously unseal thigns, then I am going to guess that a lot of the controls which I like to derive would be sealed and I’d end up having to use some sort of delegation.

    Nice Post;-)

  25. Marcel Popescu says:

    "If it is intended to be accessible to internal subclasses, then what is the harm of it being accessible to internal non-subclasses?  You get NO additional security by going to "protected and internal" because your coworker can simply write a subclass to party on the member."

    Ahem.

    double d = 5.0;

    int i = d;

    Why isn’t the above allowed? You can simply write

    int i = (int)d;

    The whole point is to force you to cast it, so that you have to think if that’s what you want. Same thing with writing a subclass to allow access to a "protected AND internal" member – it means that you thought about it and decided that yes, you really really want access to that member.

    "Not a high enough priority" is a good argument. "Not useful" is a lousy one.

  26. Rather than place the links to the most recent C# team content directly in Community Convergence , I

  27. JD says:

    Customer feedback is right on this one. ‘protected internal’ should be FamilyAndAssembly.

    The main paractical effect of these attributes is to give compiler errors when someone does something wrong. So do I want to give a compiler error to my coworker who is not derived from my class? Yes.

    You can argue that it also affects CAS and verification, and that is a very powerful argument. But by the measure of security, the C# team got the decision wrong. Because nearly everyone adding ‘protected internal’ is expecting it to be restrictive, and is surprised to learn of the contrary *if* they check their assumptions. So C# has undermined (not helped) CAS by making it more likely for developers to introduce vulnerabilities in their code.

    I would cut some slack based on design goals though. Because C# was clearly built to work well (better!) for developers with Java and C++ backgrounds. In Java is there a "package protected" (I honestly have no clue)? In C++ there aren’t really sassemblies as such, but can ‘friend’ access ‘portected’ members (since they can access privates, I assume so).

    To put it a different way, who does the current ‘protected internal’ help? It only helps non-subclassed code in the assembly have access to protected members. This is trivially possible (and arguably better modeled) by adding a regular internal method anyway. Maybe Java "protected package" did soemthing like this and it was a competitive response? I really don’t understand how this made the bar to get into the language spec as is.

    Honestly, from reading your article it sounds like there was some champion who really believed the ‘access modifiers are a loosening of the private constraint’ and no one else cared enough to argue back strongly.  

  28. Timwi says:

    I don’t entirely agree with your analysis of people’s perception. I don’t think people conceive of "internal", "protected" and "private" as restrictions because they think of "public" as some sort of "natural" state. I think they conceive of them as restrictions because the words themselves emphasise that they are a restriction; especially "protected" and "internal". They both sound much more like "stop someone from doing something" than "allow someone to do something".

    I have no idea what they could have been called instead, but this explains why the "wrong" intuition is so common.

  29. Leat says:

    It's a common belief that you cannot make some members both protected AND internal.

    And its true that you cannot do so in a single line, as many, including myself, would wish, but with some cleverness it is 100% do-able. I'll Let my (hopefully) self-documenting code speak for itself.

    [START C# CODE]

    //Code below is 100% tested

    /* FROM ProtectedAndInternal.dll */

    namespace ProtectedAndInternal

    {

       public class MyServiceImplementationBase

       {

           protected static class RelevantStrings

           {

               internal static string AppName = "Kickin' Code";

               internal static string AppAuthor = "Scott Youngblut";

           }

       }

       public class MyServiceImplementation : MyServiceImplementationBase

       {

           public void PrintProperties()

           {

               // WORKS PERFECTLY BECAUSE SAME ASSEMBLY!

               Console.WriteLine(RelevantStrings.AppAuthor);

           }

       }

       public class NotMyServiceImplementation

       {

           public void PrintProperties()

           {

               // FAILS – NOT THE CORRECT INHERITANCE CHAIN

               // Error CS0122: 'ProtectedAndInternal.MyServiceImplementationBase.Relevant' is inaccessible due to its protection level

               // Console.WriteLine(MyServiceImplementationBase.RelevantStrings.AppAuthor);

           }

       }

    }

    /* From AlternateAssemblyService.dll which references ProtectedAndInternal.dll */

    namespace AlternateAssemblyService

    {

       public class MyServiceImplementation : MyServiceImplementationBase

       {

           public void PrintProperties()

           {

               // FAILS – NOT THE CORRECT ASSEMBLY

               // Error CS0117: 'ProtectedAndInternal.MyServiceImplementationBase.RelevantStrings' does not contain a definition for 'AppAuthor'

               // Console.WriteLine(RelevantStrings.AppAuthor);

           }

       }

    }

    [END C# CODE]

    [CopyRight – Scott Youngblut (MCPD)]

  30. Leat says:

    Forgot to mention that there is a way to make members virtual, I have never had a need to do it, but it does work using delegates, Function<T>, Action<T>, and poor-man's dependency injection (http://www.lostechies.com/…/how-not-to-do-dependency-injection-in-nerddinner.aspx)

    If you NEED the sample email me, its substantially longer than this snippet.

    var email = string.Concat( @"leat" + "hakkor", "@", "hot", "mail", @".", "com" );

    ^^ Take that spammers

    [Scott Youngblut (MCPD)]

  31. Timwi says:

    > You get NO additional security by going to "protected and internal" because your coworker can simply write a subclass to party on the member.

    Isn’t this the same argument as saying, you get no additional security by going “private” because your coworker can simply write a public method to party on the member? If you think this is a strawman, please elucidate.

  32. Roman says:

    > If the member is already marked as internal then the only people who can use it are your coworkers.

    It is clear that C# has been designed around this concept, and that's fair enough. I do find it limiting though. Oftentimes I want to protect my own code _from myself_ misusing it by accident. I want to be able to write bullet-proof code that I _cannot_ possibly abuse – even if I don't have malicious intent.

    Unfortunately, in C#, if the code consists of at least two classes interacting with each other then almost always the only way to make it bullet-proof to accidental abuse is to stick it into a separate assembly.