Provisioning site collections using SP App model in on-premises with just CSOM

Back in last January, I wrote a blog post on the model how you can create site collections remotely using app model in on-premises. At the time that required still additional extension to be deployed to the on-premises farm as a farm solution. During SharePoint Conference 2014 in Las Vegas, we also did talk about the upcoming capability on doing this natively using CSOM without any full trust code to be deployed to the on-premises farm – check Rob Howard’s session on new developer APIs and features for details.

This capability was introduced to on-premises farm in the  April 2014 Cumulative Update for the SharePoint 2013 (released on 7th of May 2014) and you’ll have to explicitly enable the capability in the farm to be able to use it. This blog post is written for sharing the the step by step guidance on the required actions for using just CSOM from provider hosted apps or from any other source (like PowerShell or console apps) to create site collections remotely in on-premises farm. I’ve also included some generic content related on the site collection provisioning patterns and models, which apply also directly to the Office365.

Update on 15th of August 2014 – Updated with additional guidance on supported site templates when you use publishing sites as the tenant admin site. Guidance included later on the document with Updated marker. Also updated the code to point to the Office 365 Developer Patterns and Practices GitHub project, where we have latest versions of the code available.

Update on 14th of September 2015 – Notice that you can pass LCID for the created site collection when using SiteCreationProperties class. If this property is not set, SharePoint will use SPContext.Current.Site.RootWeb.Locale.LCID from the tenant admin site collection where the context was established. If you explicitly provider LCID fro new site collection, it will be used. 

Requirements

There are some pre-requirements to enable site collection provisioning with CSOM in on-premises farm. Reasoning for these steps is that we’ve tried to ensure minimum difference between on-premises and Office365 development patterns and there was no point introducing new CSOM API which are specifically for the on-premises after the introduction of site collection creation capability to the Offic365. Here’s the required steps one by one. Notice that the PowerShell scripts are also included in the downloadable sample code.

  1. Install April 2014 Cumulative Update for the SharePoint 2013 to your SharePoint 2013 farm
  2. Enable remote site collection creation using just CSOM by running specific PowerShell script available below
  3. Download the Windows PowerShell for SharePoint Online and install that to your remote development environment where you write your CSOM code
    • This contains 15 version of the objects which are also needed with SP2013 on-premises. These are located in the Microsoft.Online.SharePoint.Client.Tenant.dll assembly, which is by default installed to C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell folder.
    • Latest version is from May 2014, but the previous released version from 2012 does also work. since we have not introduced any real changes on the client side of the code.
  4. Enable AdministrationSiteType property from one site collection in your on-premises farm which will be acting like a admin site for site collection creation. This could be for example root site in the root site collection of the application where you create the new site collections.

Here’s the needed PowerShell for the step 2. This will enable the site collection creation in the on-premises farm with CSOM.

 #
 # Enable the remote site collection creation for on-prem in web application level
 # If this is not done, unknon object exception is raised by the CSOM code
 #
 $WebApplicationUrl = https://dev.contoso.com
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null) 
 {
     Write-Host "Loading SharePoint Powershell Snapin"
     Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }    
  
 $webapp=Get-SPWebApplication $WebApplicationUrl
 $newProxyLibrary = New-Object "Microsoft.SharePoint.Administration.SPClientCallableProxyLibrary"
 $newProxyLibrary.AssemblyName = "Microsoft.Online.SharePoint.Dedicated.TenantAdmin.ServerStub, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
 $newProxyLibrary.SupportAppAuthentication = $true
 $webapp.ClientCallableSettings.ProxyLibraries.Add($newProxyLibrary)
 $webapp.Update()
 Write-Host "Successfully added TenantAdmin ServerStub to ClientCallableProxyLibrary."
 # Reset the memory of the web application
 Write-Host "IISReset..."    
 Restart-Service W3SVC,WAS -force
 Write-Host "IISReset complete on this server, remember other servers in farm as well."    
  

Here’s the needed PowerShell for the step 4 to mark one site collection used during provisioning as the admin site. This is just due usage of the same CSOM objects as we use with the Office365. Setting the SPAdministrationSiteType to the site collection does not really impact the environment in any other ways. Site can be actually just oob team site or any other type and we’ll just need to update this property. If you need to provision host named site collections, also this Tenant admin site have to be host named site collection.

 #
 # Set admin site type property to the site collection using PS for any site collection type.
 # This is needed to be set for the site collection which is used as the 
 # "Connection point" for the CSOM when site collections are created in on-prem
 #
 $siteColUrl = https://dev.contoso.com
  
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null) 
 {
     Write-Host "Loading SharePoint Powershell Snapin"
     Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }
  
 $site = get-spsite -Identity $siteColUrl
 $site.AdministrationSiteType = [Microsoft.SharePoint.SPAdministrationSiteType]::TenantAdministration
  

Architecture design for remote provisioning

Here’s a logical architecture design for the remote provisioning of the site collections from provider hosted app side. With the new support for the on-premises, we don’t need to install any additional solutions to the SharePoint side and all code is running in the provider hosted app side.

image

  1. UI for the end users to select the type of the template and to typically set additional metadata for the created site
  2. Detailed configuration and options are dependent on the provider hosted app side. Typically we have configurations to provide theme or custom master page to the selected template together with some additional elements
  3. Actual provisioning of the site collection using CSOM. You can use CSOM or REST for additional configurations

Above model is good logical structure, but it’s not really optimal from end user perspective, since unless we make the provisioning asynchronous, end user has to wait on the page while the provisioning is happening. This is definitely not optimal, so typically in production these models are implemented using asynchronous patterns, which are completely doable with the app model as well. Here’s a reference process of the model where the site collection requests are processed asynchronously.

image

  1. When new site collection is created, users are redirected to specific provider hosted apps for filling the needed information
  2. Request of the new site collection is stored to specific list located for example in root site collection of the web application or tenant (if in Office365)
  3. You can associate separate approval workflow to the submissions if that’s needed simply by using out of the box approval workflow model or just as well you could create new workflow for more complex scenarios using SharePoint designer or any other means
  4. Approved requests are processed by remote timer job, which is configured to check the requests from the specific list. There are multiple ways to schedule the needed code for this, like using worker process or web job in Microsoft Azure. In on-premises you could run this as scheduled windows task or execute it from PowerShell as needed.
    • Check for example the \\Samples\Core.SimpleTimerJob reference example from the Office AMS v2 package for additional details on the implementation details
    • Note. for high availability purposes you’d simply have “in progress” stamp in the list for marking the items which are taken into processes by scheduled tasks from different servers running the remote timer job. This way you have high availability for request processing even in on-premises.
  5. Actual provisioning of the site collections is performed based on the stored data in the list using CSOM.
  6. When site provisioning is completed, notification is sent for the requestor using email. You can also for example show the latest newly created site collections in front page of the intranet, if that’s what business is looking for as a additional capability with the email… or push notifications to social feeds. Whatever is needed.

In production another consideration is that you really don’t want to place your remote site collection calling code directly to the provider hosted app code, since from agility and code reuse perspective you actually want to locate the code to centralized assembly, which can be then used from any solution, like from provider hosted app, PowerShell script or for example from console application. This way you can reuse the code and remote operations also for example for pushing branding updates to previously provisioned site collection. It’s not just the initial provisioning, we’ll need to remember the future updates as well.

image

  1. Provider hosted app running the synchronous operations for the end user
  2. Provider hosted app configuration and the assets which are being pushed to newly created site collections
  3. PowerShell based model alternative to call the site collection provisioning without web UI. PowerShell is just one example, just as well this could be console application running as WebJob in Windows Azure in the cloud
  4. Actual business logic for provisioning of the site collections is stored in separate isolated component for promoting code reuse and to enable the same capabilities to be executed from multiple different solutions. This enabled also easier update of the existing site collections just by calling the provisioning logic against the existing structures from PowerShell or remote timer job side
  5. SharePoint which we don’t have to add any customizations and we don’t need to have access to the servers either since code is executed from out side of the SharePoint and we can push all the needed settings

Show me the code

Actual code for remote calling looks actually pretty much the same as for creating site collections to Office365 using the same objects from online assembly. You’ll need to have reference to the 15 version of the Microsoft.Online.SharePoint.Client.Tenant.dll assembly. After this is done, only few lines of code to actually provision the new site collection. Here’s as simple as it gets example for the site creation using app only token.

 private string CreateSiteCollection(string hostWebUrl, string url, string template, string title, string adminAccount)
 {
     // Resolve root site collection URL from host web. We assume that this has been set as the "TenantAdminSite"
     string rootSiteUrl = hostWebUrl.Substring(0, 8 + hostWebUrl.Substring(8).IndexOf("/"));
  
     //Resolve URL for the new site collection
     var webUrl = string.Format("{0}/sites/{1}", rootSiteUrl, url);
  
     // Notice that this assumes that AdministrationSiteType as been set as TenantAdministration for root site collection
     // If this tenant admin URI is pointing to site collection whihc is host named site collection, code does create host named site collection as well
     var tenantAdminUri = new Uri(rootSiteUrl);
     string realm = TokenHelper.GetRealmFromTargetUrl(tenantAdminUri);
     var token = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, tenantAdminUri.Authority, realm).AccessToken;
     using (var adminContext = TokenHelper.GetClientContextWithAccessToken(tenantAdminUri.ToString(), token))
     {
         var tenant = new Tenant(adminContext);
         var properties = new SiteCreationProperties()
         {
             Url = webUrl,
             Owner = adminAccount,
             Title = title,
             Template = template
         };
  
         //start the SPO operation to create the site
         SpoOperation op = tenant.CreateSite(properties);
         adminContext.Load(op, i => i.IsComplete);
         adminContext.ExecuteQuery();
     }
     return webUrl;
 }

Notice also that unlike with Office365, this operation is completely synchronous, meaning that we move the following lines of code only after the site collection has been truly created. This means that you should be rather implement the creation of the site collection using remote timer job, than creating this directly from the provider hosted app.

Update on 15th of August – We have found one additional requirement related on the used web template which you will set for the newly created site collection. Site which is used as the tenant admin site will actually control the available templates which you can use for the newly created site collection. This is evident if you have used publishing site template for the tenant admin site, which would limit the default support site templates to be only publishing sites templates. In that case you’d have to set the additional templates supported as the sub site templates for the tenant site to make this work. Alternatively if you have deployed custom web template, you will have to also make sure that it’s available for the tenant admin site.

Video recording of the demo code

Here’s quick video on showing the required steps and walkthrough the code for the sample provider hosted app, which is provided as downloadable sample. Notice that in my on-premises lab I’m using the low trust oAuth model (MSDN guidance updated on 2nd of June 2014), which means that there’s no need for S2S (high trust) setup between the app and SharePoint farm. Code will work also as such with the S2S app authentication model.

Questions and answers

Here’s few questions which I’d ask after reading above blog post to get some additional details on supported capabilities and how things actually work. I’ll update this if needed based on the blog comments.

Question – “Why do we need to explicitly enable this capability in on-premises farms after installation of April CU?”
Answer – I might comment that it’s due carefully though process where we want administrators to control this capability in the farm. Reality however could be also that we forgot the needed step in April CU and the additional step will be included in future CU updates. Whatever is the actual truth, these are the steps to enable the capability after April CU. 

Question – “My customer is using normal Office365, is this the guidance for creating site collections in there as well?”
Answer – Architecture guidance and the overall process is pretty similar, but the actual code is purely targeted to on-premises. In the Office365, you can use the normal Tenant admin objects. Check for example Richard’s blog post for additional information.

Question – “My customer is using Office365 Dedicated (not normal Office365) as their platform. Will this work in dedicated platform as well?”
Answer – Yes. This will be available in the Dedicated environment when the farms are upgraded to the April CU level. You should contact your Microsoft SDM to check the schedule in detail for your environment..

Question – “I need to apply custom branding to the created site collection, how do I do that? Should I used web templates or something like that?”
Answer – This code only creates the site collection and sets a oob theme for it, but you could just as well continue by applying custom branding using CSOM. Recommended approach would be simply to use CSOM to upload the needed assets to the site collection and for creating the site columns and content type. These models are explained in details at the Office AMS samples.

Question – “Does this model also work if I used host named site collections in on-premises?
Answer – Ye, this was also closely tested and verified. This works as long as the site which is acting as the Tenant Administration Site is also host named site collection, meaning that the code looks exactly the same for path based and host named, but you can control this using the type of the Tenant Administration Site.

Question – “Can I set the quota template for the site during provisioning?
Answer – No. This option is not available from the CSOM side. You have to control this from the central administration. There’s no CSOM to change or select the used quote template at this time.

Question – “Can I control to which database is created using CSOM?
Answer – No. This option is not exposed in CSOM creation.

Question – “ Can I use these new objects for on-premises to list existing site collections or to check if URL has been already in use when site collection request is done?”  
Answer – Unfortunately not right now. This could be indeed nasty limitation which will be addressed sooner or later, but right now you’d have to use search APIs to solve remotely what site collections currently exists in the SharePoint.

Question – Why would I use app pattern in on-premises?
Answer – Your call. Full trust code is fully supported in the on-premises as well and it is completely your call if you want to use the app model in on-premises or not. I personally would like us to get closer to the situation where the on-prem or cloud is not exception, we would rather use the same model in both sides. There are obviously major differences on the both sides (on-premises and cloud) right now and capabilities are not the same on both sides. This is not definitely optimal, but just the situation as it is right now. Using app model or full trust code is completely your call based on the business objective. We recommend the app model where possible for aligning to product and service roadmap due obvious (also selfish) reasons, but full trust code is also fully supported in on-premises.
(note. this final answer was updated after initial version based on valid community feedback on the wording. Thanks Spence for your input.)

References

Some references related on the topics covered in this blog post.