SharePoint, Claims, WS-* and the Windows Identity Foundation (WIF)

SharePoint Claims Authentication Series

01 - SharePoint, Claims, WS-* and the Windows Identity Foundation (WIF)
02 - WS-Federation Authentication Module and SharePoint Extensions
03 - The SharePoint Federation Authentication Module
04 - Integrating SharePoint with Windows Azure Active Directory (ACS)
05 - Session Authentication Module and WS-SecureConversation
06 - SharePoint Session Authentication Module
07 - Windows Claims Authentication
08 - Service (ActAs) Tokens

Thank you to Steve Peschka for reviewing these articles!

SharePoint Claims Authentication Part 1

SharePoint 2010 introduced Claims authentication based on the WS-Trust, WS-Federation, and SAML standards, bringing new opportunities for federation, authentication, and authorization scenarios. It also brought a brave new world of concepts and patterns for SharePoint administrators and developers to learn and master. In this series of articles, I'll take you on a deep dive through the mechanisms of the Windows Identity Foundation (WIF) Claims authentication pipeline with a special focus on its application to SharePoint. By understanding these from the inside out I hope you'll be better prepared to implement, maintain, extend and troubleshoot SharePoint, Azure, and O365 integration and federation and maximize your organization's investments in these and other products. As we progress I hope you'll not only find yourself better understanding Claims authentication within SharePoint, but also better understanding Claims authentication in general.

Why Claims?

You may be forgiven at this point for asking why. Kerberos has been a relatively stable protocol for many years and is still supported in SharePoint and other Windows applications. NTLM, warts and all, continues to function effectively in many environments. Nevertheless, several factors should drive you to seriously consider Claims authentication.

First, Kerberos and NTLM haven't been significantly updated in many years. Claims and complementary security protocols such as OAuth, OpenID and SAML offer new ways to delegate authentication, authorization, and user profile and attribute information in ways not possible with only the older protocols. Second, NTLM and Kerberos in Windows are not particularly well-suited for interoperation with other applications and systems. Within a single organization Windows authentication alone may be effective; but when federating with other organizations and working across the Internet, SAML and other Claims-based forms of authentication are a more common and supported option. Even within organizations, Claims significantly improves interoperability and coordination between Windows applications such as SharePoint and non-Windows applications and systems. For example, SharePoint's Business Connectivity Services leverage the interoperability made possible by Claims authentication to allow data from external LOB applications to be integrated with SharePoint.

Finally, Claims Authentication is not only the wave of the future, it's already well-established in the present. Microsoft and non-Microsoft systems and services alike utilize and offer Claims-based authentication already. Windows Azure Active Directory (aka Azure Access Control Service) and Active Directory Federation Services (ADFS) are based on Claims authentication. In .NET 4.5, the Claims Principal will become the base identity for all threads (as detailed in Vittorio Bertocci's blog post here). On the Internet, when you sign in to Windows Live ID or grant access to your Facebook profile you are utilizing WS-Federation and OAuth. To allow your applications, SharePoint included, to interoperate with all these other systems and services, using and understanding Claims authentication is a necessity.

The Claims Authentication Stack

Claims authentication is a process involving many participants and components. While many blog posts and technical articles are available to help you set up Claims authentication in SharePoint and other applications, my goal in this series is to take you on a step-by-step journey through all the components and protocols involved. If I miss something or you disagree with the way I describe things let me know in the comments and I'll respond or amend the content as appropriate. Occasionally I may leave out a detail or two to keep us focused on the fundamentals, but as you'll discover my hope is to share as much as possible. We'll focus primarily on the Claims authentication stack provided by Windows Identity Foundation (WIF), especially as it is extended and applied in SharePoint.

The foundation of the Claims authentication process in WIF and other authentication stacks is a number of standards and specifications managed by the Organization for the Advancement of Structured Information Standards (OASIS). These include the following:

WS-Security: Security Tokens

All of these standards are based on the concept of a security token, defined in WS-Security as follows:

A security token represents a collection (one or more) of claims. (WSS: SOAP Message Security, Page 11, Line 326)

Well-known types of security tokens include X.509 certificates, Kerberos tickets, SAML tokens, and, yes, Username/Password tokens. The claims asserted in a security token can include subjects, attributes (e.g. email addresses, names, and identifiers), symmetric and asymmetric (public) keys, passwords, and more. These claims often form the basis for authentication and authorization decisions, and may also be utilized to populate profiles or sign other tokens and messages.

Let's take an X.509 certificate token as an example. It specifies claims including the name of an entity and a public key associated with that entity. If a receiver of the X.509 token trusts this information, it can utilize the public key to verify that other messages signed by the associated private key truly originated from the associated entity. In addition, X.509 certificates are signed by another security token; this signature represents another claim asserted by the signing X.509 certificate that the entity in the signed certificate is the proper owner of the included public key.

To be useful, tokens received from a third party are almost always signed. The WS-Security spec defines a signed token as follows:

A signed security token is a security token that is asserted and cryptographically signed by a specific authority (e.g. an X.509 certificate or a Kerberos ticket). (WSS: SOAP Message Security, Page 12, Line 330)

However, just because a token has been signed by its issuer doesn't mean that the party presenting it to some other application is the proper owner and user of the token. For example, it could have been stolen in transit. To help confirm that the bearer of a token is in fact the proper owner of that token, some tokens, such as X.509 certificates and Kerberos tickets, associate the owner with a security key known only to that owner (e.g. a private key). This allows receivers to guarantee that messages signed by the specified private key originate from the proper owner of the token by checking that these messages have been signed by the associated key.

On the other hands, some tokens, notably SAML bearer tokens, don't rely on the bearer possessing a key; proof of ownership is demonstrated simply through possession. This is similar to cookie-based authentication, where the possessor of the cookie is assumed to be its rightful owner. Of course, HTTPS or some other transport-level encryption is essential in guaranteeing that the token or cookie hasn't been stolen. The concept of a bearer token comes up a lot in current Claims authentication, so come back and reread these last couple paragraphs in the future if you need to.

In the Windows Identity Foundation, Security Tokens are created, read, and validated by SecurityTokenHandlers. Pre-programmed handlers available out-of-the-box include a Kerberos handler, a SAML1.1 handler, a SAML2 handler, and a X509 handler, among others. SharePoint only supports handling SAML1.1 tokens directly.

WS-Trust: Requesting and Receiving Security Tokens

Having defined a security token, we now proceed to define how one goes about acquiring one. In some cases, such as X.509 certificates and Kerberos tickets, the token is acquired via "legacy" protocols such as via a Kerberos Ticket Granting Service or a X.509 Certificate Authority. For newer token formats such as SAML tokens, WS-Trust provides a protocol for requesting and receiving a token. In WS-Trust, the requestor sends a RequestSecurityToken (RST) request to a Security Token Service (STS) asking that STS to issue a security token asserting specified claims. Once the STS has satisfied itself that the requestor is entitled to the requested claims, it responds with a RequestSecurityTokenResponse (RSTR) containing the requested security token and any associated keys needed to use it.

For example, an entity with a Windows identity may request a SAML token specifying their UPN and information about AD groups they are part of. The requesting entity would craft an RST specifying the desired claims. It would then create a SOAP message including the RST in the body and a Kerberos ticket token in the header (to prove its rights to these claims) and submit this message to an STS. The STS would verify the Kerberos token and then create a SAML token with the requested claims. It would place the SAML token in an RSTR and place the RSTR in a SOAP message, which it would then send back to the requesting entity. The requesting entity would extract the new SAML security token from the message and use it as needed.

The Security Token Service provided with WIF and the STS implemented by SharePoint as the Security Token Service Application are both WS-Trust STS's. That is, they expect and process RSTs submitted directly in SOAP-based Issue requests, and send RSTRs back to the requestors in SOAP messages. (Note: the WIF STS can also act as a WS-Federation endpoint; more on that soon.)

Another common protocol for requesting and receiving security tokens is SAMLP (SAML Protocol). Although SAMLP is part of the SAML2 specification, you should be careful not to confuse SAML tokens with the SAML protocol. SAML tokens are XML security tokens asserting a collection of claims. The SAML protocol (SAMLP) is a protocol for requesting SAML (or other) security tokens. Currently, WIF and SharePoint do not support SAMLP, although ADFS does.

By understanding WS-Security and WS-Trust we've come a long way, but one element remains to be explained. The WS-Trust protocol just described works well in scenarios where an entity knows how to actively request and process a security token. This is fine for service-to-service communication, where the requesting service can be programmed to do just that. Web browsers, however, do not know how to craft RSTs and process RSTRs and security tokens. So another layer is needed to define how to request and utilize security tokens in end-user web applications (e.g. SharePoint). This brings us to…

WS-Federation: Web Browsers (i.e. Passive Requestors)

WS-Federation, specifically Section 13, contains a profile describing how to utilize the WS-Trust request/response protocol via a web browser, also referred to as a "passive" requestor. The simplest form of the process is illustrated in the following image from the spec:

(Web Services Federation Language v1.2, page 95, Line 3173)

When the browser requests a resource (e.g. a web page) without the required token (1), the web application (also known as a Relying Party, or RP) redirects the browser to an STS to get the needed token. It does this by replying to the request with a 302 Redirect response specifying the address of the STS endpoint to which the client should connect (2). The web application appends details about the token needed by adding a query string to this redirect URL. The web browser follows the redirect and sends the URL with query parameters to the STS endpoint (3). The endpoint authenticates the end-user (without going into details, this could be via a Kerberos ticket or a username/password combination) (4) and then prepares a RSTR to send back to the original web page. Here again, the browser cannot handle the RSTR on its own. Instead, the STS needs to redirect the browser with the appropriate RSTR back to the original web page (5). A 302 Redirect isn't enough in this case, as 302s can only perform GET requests to the destination URL and an RSTR is often too big to fit in a query string. A POST request to the original RP is initiated by setting some script in the response at step 5 that posts the RSTR to the original web page via the browser (6).

If all goes well, at this point the original web application will have an RSTR from the STS. It (the web application) can extract the security token from the RSTR and use it to authenticate the user and send back the requested resource (7).

Following along with the passive requestor profile

To help clarify this, let's follow the process when I access the management page for a namespace of mine in Windows Azure Active Directory (Azure Access Control Service). The management page is at https://joshgav.accesscontrol.windows.net/v2/mgmt/web. When I browse to this address in Internet Explorer I receive the following response:

HTTP/1.1 302 Found
Location: https://joshgav.accesscontrol.windows.net/v2/wsfederation?**wa**=wsignin1.0&**wtrealm**=https%3a%2f%2fjoshgav.accesscontrol.windows.net%2fv2%2fmgmt%2fservice&**wreply**=https%3a%2f%2fjoshgav.accesscontrol.windows.net%3a443%2fv2%2fmgmt%2fweb%2fsignin&**wctx**=%2fv2%2fmgmt%2fweb

I've highlighted the query parameter keys to make it somewhat easier to identify them. These parameters are all defined in the WS-Federation spec. wa stands for the Action requested; wsignin1.0 and wsignout1.0 are the two most common actions and represent sign-in and sign-out requests respectively. wtrealm stands for Trusting Realm and designates the security realm which the requested security token is intended for. wreply specifies where the user should be sent back to with the received RSTR. Note that most of the time this reply address is the same or a child of the Trust Realm (wtrealm) (and often only or the other is specified). wctx is an optional parameter for specifying context to be retained and replayed back to the original page in the reply.

The URL I'm being redirected to (https://joshgav.accesscontrol.windows.net/v2/wsfederation) is the endpoint for the WS-Federation passive STS endpoint for my own ACS domain. ACS, WIF, and ADFS can all provide a WS-Federation STS endpoint. This STS is now responsible for authenticating me and ultimately sending me back to the URL and page I originally requested. It could authenticate me with a simple name and password or numerous other options which we won't get into right now. Once I'm authenticated, it sends me back a page with the requested RSTR embedded in a form along with some JavaScript directing my browser to post this form back to the original URL. The returned page looks like this:

<html>
<head>
<title>Working...</title>
</head>
<body>
<form method="POST" name="hiddenform" action="https://joshgav.accesscontrol.windows.net/v2/mgmt/web/signin">
<input type="hidden" name="wa" value="wsignin1.0" />
<input type="hidden" name="wresult" value="HTML-encoded RSTR goes here (decoded below) " />
<input type="hidden" name="wctx" value="/v2/mgmt/web" />
<noscript><p>Script is disabled. Click Submit to continue.</p><input type="submit" value="Submit" /></noscript>
</form>
<script language="javascript">window.setTimeout('document.forms[0].submit()', 0);</script>
</body>
</html>

The RSTR goes where I've indicated in bold. If you'd like to see what an RSTR looks like, I decoded and included it below. Note that ACS uses a Simple Web Token (SWT) security token and not a SAML token; we won't worry about that right now. I took the liberty of converting the SWT from Base64 and URL decoding it so you'd have the pleasure of seeing what it actually contains. This is a bearer token, by the way. If it was still valid (and I hadn't modified the HMAC), you could take it right out of this blog post and authenticate to my management service with it.

<t:RequestSecurityTokenResponse Context="/v2/mgmt/web" xmlns:t="https://schemas.xmlsoap.org/ws/2005/02/trust">
<t:Lifetime>
<wsu:Created xmlns:wsu="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2012-04-25T20:07:47.376Z</wsu:Created>
<wsu:Expires xmlns:wsu="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2012-04-26T04:07:47.376Z</wsu:Expires>
</t:Lifetime>
<wsp:AppliesTo xmlns:wsp="https://schemas.xmlsoap.org/ws/2004/09/policy">
<EndpointReference xmlns="https://www.w3.org/2005/08/addressing">
<Address>https://joshgav.accesscontrol.windows.net/v2/mgmt/service</Address>
</EndpointReference>
</wsp:AppliesTo>
<t:RequestedSecurityToken>
<wsse:BinarySecurityToken
wsu:Id="uuid:73344c18-47a2-40d7-9ea7-1b63a8eb182c"
ValueType="https://schemas.xmlsoap.org/ws/2009/11/swt-token-profile-1.0"
EncodingType="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary"
xmlns:wsu="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
xmlns:wsse="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" >
https://schemas.microsoft.com/ws/2008/06/identity/claims/role=Administrator
&https://schemas.microsoft.com/accesscontrolservice/2010/07/claims/identityprovider=uri:WindowsLiveID
&Audience=https://joshgav.accesscontrol.windows.net/v2/mgmt/service
&ExpiresOn=1336035077&Issuer=https://joshgav.accesscontrol.windows.net/
&HMACSHA256=e3JilQ6AHWCH5FtSwqXxEEgQ6mUF6gOa1PvoyAi65fI=
</wsse:BinarySecurityToken>
</t:RequestedSecurityToken>
<t:TokenType>https://schemas.xmlsoap.org/ws/2009/11/swt-token-profile-1.0</t:TokenType>
<t:RequestType>https://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>
<t:KeyType>https://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>
</t:RequestSecurityTokenResponse>

Once this form with its keys and values is posted back to the original requested URL (the ACS management page), the web application extracts the presented token, checks my authorization for the page requested and (if I'm authorized) serves me said page. In the case of the ACS Management service and most web apps, a cookie is set at this point so that the token won't need to be requested and/or processed for each subsequent request.

Conclusion

At this point we've covered the WS-Security, WS-Trust, and WS-Federation protocols. We've touched on SAML and other token profiles such as Kerberos, X.509, and even SWT. There's still quite a long ways to go in explaining how WIF automates and coordinates all of these elements for you, but with these foundations you'll find the ensuing conversations easier to follow.