Provisioning site collections using SP App model in on-premises with just CSOM


Back in last January, I wrote a blog post on the model how you can create site collections remotely using app model in on-premises. At the time that required still additional extension to be deployed to the on-premises farm as a farm solution. During SharePoint Conference 2014 in Las Vegas, we also did talk about the upcoming capability on doing this natively using CSOM without any full trust code to be deployed to the on-premises farm – check Rob Howard’s session on new developer APIs and features for details.

This capability was introduced to on-premises farm in the  April 2014 Cumulative Update for the SharePoint 2013 (released on 7th of May 2014) and you’ll have to explicitly enable the capability in the farm to be able to use it. This blog post is written for sharing the the step by step guidance on the required actions for using just CSOM from provider hosted apps or from any other source (like PowerShell or console apps) to create site collections remotely in on-premises farm. I’ve also included some generic content related on the site collection provisioning patterns and models, which apply also directly to the Office365.

Update on 15th of August 2014 – Updated with additional guidance on supported site templates when you use publishing sites as the tenant admin site. Guidance included later on the document with Updated marker. Also updated the code to point to the Office 365 Developer Patterns and Practices GitHub project, where we have latest versions of the code available.

Update on 14th of September 2015 – Notice that you can pass LCID for the created site collection when using SiteCreationProperties class. If this property is not set, SharePoint will use SPContext.Current.Site.RootWeb.Locale.LCID from the tenant admin site collection where the context was established. If you explicitly provider LCID fro new site collection, it will be used. 

Requirements

There are some pre-requirements to enable site collection provisioning with CSOM in on-premises farm. Reasoning for these steps is that we’ve tried to ensure minimum difference between on-premises and Office365 development patterns and there was no point introducing new CSOM API which are specifically for the on-premises after the introduction of site collection creation capability to the Offic365. Here’s the required steps one by one. Notice that the PowerShell scripts are also included in the downloadable sample code.

  1. Install April 2014 Cumulative Update for the SharePoint 2013 to your SharePoint 2013 farm
  2. Enable remote site collection creation using just CSOM by running specific PowerShell script available below
  3. Download the Windows PowerShell for SharePoint Online and install that to your remote development environment where you write your CSOM code
    • This contains 15 version of the objects which are also needed with SP2013 on-premises. These are located in the Microsoft.Online.SharePoint.Client.Tenant.dll assembly, which is by default installed to C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell folder.
    • Latest version is from May 2014, but the previous released version from 2012 does also work. since we have not introduced any real changes on the client side of the code.
  4. Enable AdministrationSiteType property from one site collection in your on-premises farm which will be acting like a admin site for site collection creation. This could be for example root site in the root site collection of the application where you create the new site collections.

Here’s the needed PowerShell for the step 2. This will enable the site collection creation in the on-premises farm with CSOM.

#
# Enable the remote site collection creation for on-prem in web application level
# If this is not done, unknon object exception is raised by the CSOM code
#
$WebApplicationUrl = http://dev.contoso.com
$snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
if ($snapin -eq $null) 
{
    Write-Host "Loading SharePoint Powershell Snapin"
    Add-PSSnapin "Microsoft.SharePoint.Powershell"
}    
 
$webapp=Get-SPWebApplication $WebApplicationUrl
$newProxyLibrary = New-Object "Microsoft.SharePoint.Administration.SPClientCallableProxyLibrary"
$newProxyLibrary.AssemblyName = "Microsoft.Online.SharePoint.Dedicated.TenantAdmin.ServerStub, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
$newProxyLibrary.SupportAppAuthentication = $true
$webapp.ClientCallableSettings.ProxyLibraries.Add($newProxyLibrary)
$webapp.Update()
Write-Host "Successfully added TenantAdmin ServerStub to ClientCallableProxyLibrary."
# Reset the memory of the web application
Write-Host "IISReset..."    
Restart-Service W3SVC,WAS -force
Write-Host "IISReset complete on this server, remember other servers in farm as well."    
 

Here’s the needed PowerShell for the step 4 to mark one site collection used during provisioning as the admin site. This is just due usage of the same CSOM objects as we use with the Office365. Setting the SPAdministrationSiteType to the site collection does not really impact the environment in any other ways. Site can be actually just oob team site or any other type and we’ll just need to update this property. If you need to provision host named site collections, also this Tenant admin site have to be host named site collection.

#
# Set admin site type property to the site collection using PS for any site collection type.
# This is needed to be set for the site collection which is used as the 
# "Connection point" for the CSOM when site collections are created in on-prem
#
$siteColUrl = http://dev.contoso.com
 
$snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
if ($snapin -eq $null) 
{
    Write-Host "Loading SharePoint Powershell Snapin"
    Add-PSSnapin "Microsoft.SharePoint.Powershell"
}
 
$site = get-spsite -Identity $siteColUrl
$site.AdministrationSiteType = [Microsoft.SharePoint.SPAdministrationSiteType]::TenantAdministration
 

Architecture design for remote provisioning

Here’s a logical architecture design for the remote provisioning of the site collections from provider hosted app side. With the new support for the on-premises, we don’t need to install any additional solutions to the SharePoint side and all code is running in the provider hosted app side.

image

  1. UI for the end users to select the type of the template and to typically set additional metadata for the created site
  2. Detailed configuration and options are dependent on the provider hosted app side. Typically we have configurations to provide theme or custom master page to the selected template together with some additional elements
  3. Actual provisioning of the site collection using CSOM. You can use CSOM or REST for additional configurations

Above model is good logical structure, but it’s not really optimal from end user perspective, since unless we make the provisioning asynchronous, end user has to wait on the page while the provisioning is happening. This is definitely not optimal, so typically in production these models are implemented using asynchronous patterns, which are completely doable with the app model as well. Here’s a reference process of the model where the site collection requests are processed asynchronously.

image

  1. When new site collection is created, users are redirected to specific provider hosted apps for filling the needed information
  2. Request of the new site collection is stored to specific list located for example in root site collection of the web application or tenant (if in Office365)
  3. You can associate separate approval workflow to the submissions if that’s needed simply by using out of the box approval workflow model or just as well you could create new workflow for more complex scenarios using SharePoint designer or any other means
  4. Approved requests are processed by remote timer job, which is configured to check the requests from the specific list. There are multiple ways to schedule the needed code for this, like using worker process or web job in Microsoft Azure. In on-premises you could run this as scheduled windows task or execute it from PowerShell as needed.
    • Check for example the \\Samples\Core.SimpleTimerJob reference example from the Office AMS v2 package for additional details on the implementation details
    • Note. for high availability purposes you’d simply have “in progress” stamp in the list for marking the items which are taken into processes by scheduled tasks from different servers running the remote timer job. This way you have high availability for request processing even in on-premises.
  5. Actual provisioning of the site collections is performed based on the stored data in the list using CSOM.
  6. When site provisioning is completed, notification is sent for the requestor using email. You can also for example show the latest newly created site collections in front page of the intranet, if that’s what business is looking for as a additional capability with the email… or push notifications to social feeds. Whatever is needed.

In production another consideration is that you really don’t want to place your remote site collection calling code directly to the provider hosted app code, since from agility and code reuse perspective you actually want to locate the code to centralized assembly, which can be then used from any solution, like from provider hosted app, PowerShell script or for example from console application. This way you can reuse the code and remote operations also for example for pushing branding updates to previously provisioned site collection. It’s not just the initial provisioning, we’ll need to remember the future updates as well.

image

  1. Provider hosted app running the synchronous operations for the end user
  2. Provider hosted app configuration and the assets which are being pushed to newly created site collections
  3. PowerShell based model alternative to call the site collection provisioning without web UI. PowerShell is just one example, just as well this could be console application running as WebJob in Windows Azure in the cloud
  4. Actual business logic for provisioning of the site collections is stored in separate isolated component for promoting code reuse and to enable the same capabilities to be executed from multiple different solutions. This enabled also easier update of the existing site collections just by calling the provisioning logic against the existing structures from PowerShell or remote timer job side
  5. SharePoint which we don’t have to add any customizations and we don’t need to have access to the servers either since code is executed from out side of the SharePoint and we can push all the needed settings

Show me the code

Actual code for remote calling looks actually pretty much the same as for creating site collections to Office365 using the same objects from online assembly. You’ll need to have reference to the 15 version of the Microsoft.Online.SharePoint.Client.Tenant.dll assembly. After this is done, only few lines of code to actually provision the new site collection. Here’s as simple as it gets example for the site creation using app only token.

private string CreateSiteCollection(string hostWebUrl, string url, string template, string title, string adminAccount)
{
    // Resolve root site collection URL from host web. We assume that this has been set as the "TenantAdminSite"
    string rootSiteUrl = hostWebUrl.Substring(0, 8 + hostWebUrl.Substring(8).IndexOf("/"));
 
    //Resolve URL for the new site collection
    var webUrl = string.Format("{0}/sites/{1}", rootSiteUrl, url);
 
    // Notice that this assumes that AdministrationSiteType as been set as TenantAdministration for root site collection
    // If this tenant admin URI is pointing to site collection whihc is host named site collection, code does create host named site collection as well
    var tenantAdminUri = new Uri(rootSiteUrl);
    string realm = TokenHelper.GetRealmFromTargetUrl(tenantAdminUri);
    var token = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, tenantAdminUri.Authority, realm).AccessToken;
    using (var adminContext = TokenHelper.GetClientContextWithAccessToken(tenantAdminUri.ToString(), token))
    {
        var tenant = new Tenant(adminContext);
        var properties = new SiteCreationProperties()
        {
            Url = webUrl,
            Owner = adminAccount,
            Title = title,
            Template = template
        };
 
        //start the SPO operation to create the site
        SpoOperation op = tenant.CreateSite(properties);
        adminContext.Load(op, i => i.IsComplete);
        adminContext.ExecuteQuery();
    }
    return webUrl;
}

Notice also that unlike with Office365, this operation is completely synchronous, meaning that we move the following lines of code only after the site collection has been truly created. This means that you should be rather implement the creation of the site collection using remote timer job, than creating this directly from the provider hosted app.

Update on 15th of August – We have found one additional requirement related on the used web template which you will set for the newly created site collection. Site which is used as the tenant admin site will actually control the available templates which you can use for the newly created site collection. This is evident if you have used publishing site template for the tenant admin site, which would limit the default support site templates to be only publishing sites templates. In that case you’d have to set the additional templates supported as the sub site templates for the tenant site to make this work. Alternatively if you have deployed custom web template, you will have to also make sure that it’s available for the tenant admin site.

Video recording of the demo code

Here’s quick video on showing the required steps and walkthrough the code for the sample provider hosted app, which is provided as downloadable sample. Notice that in my on-premises lab I’m using the low trust oAuth model (MSDN guidance updated on 2nd of June 2014), which means that there’s no need for S2S (high trust) setup between the app and SharePoint farm. Code will work also as such with the S2S app authentication model.

Questions and answers

Here’s few questions which I’d ask after reading above blog post to get some additional details on supported capabilities and how things actually work. I’ll update this if needed based on the blog comments.

Question – “Why do we need to explicitly enable this capability in on-premises farms after installation of April CU?”
Answer – I might comment that it’s due carefully though process where we want administrators to control this capability in the farm. Reality however could be also that we forgot the needed step in April CU and the additional step will be included in future CU updates. Whatever is the actual truth, these are the steps to enable the capability after April CU. 

Question – “My customer is using normal Office365, is this the guidance for creating site collections in there as well?”
Answer – Architecture guidance and the overall process is pretty similar, but the actual code is purely targeted to on-premises. In the Office365, you can use the normal Tenant admin objects. Check for example Richard’s blog post for additional information.

Question – “My customer is using Office365 Dedicated (not normal Office365) as their platform. Will this work in dedicated platform as well?”
Answer – Yes. This will be available in the Dedicated environment when the farms are upgraded to the April CU level. You should contact your Microsoft SDM to check the schedule in detail for your environment..

Question – “I need to apply custom branding to the created site collection, how do I do that? Should I used web templates or something like that?”
Answer – This code only creates the site collection and sets a oob theme for it, but you could just as well continue by applying custom branding using CSOM. Recommended approach would be simply to use CSOM to upload the needed assets to the site collection and for creating the site columns and content type. These models are explained in details at the Office AMS samples.

Question – “Does this model also work if I used host named site collections in on-premises?
Answer – Ye, this was also closely tested and verified. This works as long as the site which is acting as the Tenant Administration Site is also host named site collection, meaning that the code looks exactly the same for path based and host named, but you can control this using the type of the Tenant Administration Site.

Question – “Can I set the quota template for the site during provisioning?
Answer – No. This option is not available from the CSOM side. You have to control this from the central administration. There’s no CSOM to change or select the used quote template at this time.

Question – “Can I control to which database is created using CSOM?
Answer – No. This option is not exposed in CSOM creation.

Question – “Can I use these new objects for on-premises to list existing site collections or to check if URL has been already in use when site collection request is done?” 
Answer – Unfortunately not right now. This could be indeed nasty limitation which will be addressed sooner or later, but right now you’d have to use search APIs to solve remotely what site collections currently exists in the SharePoint.

Question – Why would I use app pattern in on-premises?
Answer – Your call. Full trust code is fully supported in the on-premises as well and it is completely your call if you want to use the app model in on-premises or not. I personally would like us to get closer to the situation where the on-prem or cloud is not exception, we would rather use the same model in both sides. There are obviously major differences on the both sides (on-premises and cloud) right now and capabilities are not the same on both sides. This is not definitely optimal, but just the situation as it is right now. Using app model or full trust code is completely your call based on the business objective. We recommend the app model where possible for aligning to product and service roadmap due obvious (also selfish) reasons, but full trust code is also fully supported in on-premises.
(note. this final answer was updated after initial version based on valid community feedback on the wording. Thanks Spence for your input.)

References

Some references related on the topics covered in this blog post.

Comments (73)

  1. Fernando Rodríguez says:

    Works perfect :),

    thank you, i was missing the Tenant Admin server stub in my web application 🙂

  2. Riccardo says:

    Excellent article! I believe you mean SharePoint April 2014 CU in the introductory text. It is very cool to see that site creation can be done now remotely without any server site code.

  3. Fernando Rodríguez says:

    Hola again, is it possible to provision site collections in a web application that doesn't have a site with a subscription and without the "tenantadmin#0" template?

  4. Vesa Juvonen says:

    Hi Fernando,

    Requirements to provision the site collection in on-prem with the April CU CSOM is that you have one site set as the "admin site". PowerShell was included in the post and this could be oob team site… you don't need to have site which is using tenant admin site template.

    You don't need to also have subscription. Only the four steps mentioned in this blog post are required for on-prem. This means that you DON't need to use multi-tenant features to make this work.

  5. Excellent article,I will try and give you feedback.

  6. Brian says:

    Thanks for this Vesa. Been waiting for this for a while but have to say I'm disappointed with the lack of ability to set DB and quota. With the guidance from MS being single web app this make it challenging to co-locate My Sites and Team Sites as there is no way from keep them in separate DBs without server side code. Hopefully this capability will arrive in the near future??

  7. Vesa Juvonen says:

    Hi Brian,

    thanks for your feedback and I completely understand the reasoning. In on-premises we have been used to define the quota templates and the DB during provisioning time for these kind of models. We don't explicitly say that single web applications is better than multiple since there are differences like this one between them and keeping the web applications in separately would help with this.

    One web app approach is simply coming from the model what the Office365 is using and in there, database structure is abstracted away from the code level, since that's considered as a configuration which the developer should not be able to control.

    If you have one web application and you'd like to route the requests to specific databases, you could theoretically do that by using custom SPSiteCreationProvider which would be installed to the farm as full trust code. This model is explained nicely by Wictor at http://www.wictorwilen.se/sharepoint-specifying-content-database-for-new-site-collections-when-using-host-named-site-collections. This would be indeed a additional full trust code, but it would be not controlled and used by the actual functionality developers, it's rather backend modifier, which does not impact on the actual content or future of the content databases. It would be only present to route the site collection requests to specific database.

    I'm personally not expecting quota and database settings to be exposed in the CSOM in near future, but if that's something we would like to request from the product group, I'd suggest to use the officespdev.uservoice.com site for the feedback. Input from the field is closely followed by the PG based on this site.

  8. Nigel says:

    I think this is exciting stuff and will be come the defacto standard for site provisioning. Its just a pain at the moment to wire up all of the infrastructure to work together. On that subject…..

    Can I have the Office 365 domain different from the on-premises domain and different from the Azure domain ?  What changes do I need to do if this is possible ?

    Thanks

    Nigel

  9. Vesa Juvonen says:

    Hi Nigel,

    right now authenticated side of the Office365 MT sites will always use the tenantname.sharepoint.com as the URL pattern. Something like tenantnames.sharepoint.com/…/teamsite.

  10. Mahendran Govender says:

    Hi There

    Thank you! This is creating Sites correctly (after I switched the code to use the SharePointContext instead of TokenHelper)on my side but I do have a few challenges in the process:

    1. It is giving me the following error:

    An exception of type 'Microsoft.SharePoint.Client.ServerException' occurred in Microsoft.SharePoint.Client.Runtime.dll but was not handled in user code

    Additional information: Value cannot be null.

    Parameter name: siteSubscription

    The Site is however created and I can browse to it.

    Also, a question, is this not creating subsites? I was hoping to create root level site collections in my web application. Maybe I am misunderstanding the concepts somewhat?

    Thanks!!

  11. Vesa Juvonen says:

    Hi Mahendran.

    exception would be indicating that you are trying to load the Site object, which is not something that works on-premises. If you use the code precisely as provided, it works.

    For the sub sites, you would be just opening new client context to just created site collection using URL and after that you can use WebCreationInformation object for creating the needed sub sites.

  12. Yvan says:

    Saw your presentation in Las Vegas and I thought it was really cool, especially when you leave the room knowing that very soon there will be a full circle completed.

    This post, closes that circle and is awesome. Thank you for this.

  13. Shriram says:

    Hi Vesa,

    Thank you for this great post!

    We built one remote console app for provisioning Sharepoint online sites with CSOM and now we are trying to use the same console app for provisioning sites On Premise. For this we will have to do some changes in code to address both scenarios(Sharepoint Online and On Premise).

    My question is how we can identify in console app code whether we are running it for "On Premise" or for 'Sharepoint Online" ?  So that then we can apply our conditions accordingly.

    Thanks in advance!

    Br,

    Shriram

  14. Vesa Juvonen says:

    Hi Shriram,

    since authentication model is different between the Online and on-premises, you really can't resolve the target by connecting to it… this means that you'd need to configure the console to know beforehand what is the target environment. As simple as it gets, but I'd simply use app.config with custom configuration key for this one.

  15. Shriram says:

    Thanks Vesa for the quick clarification ! 🙂

    Br,

    Shriram

  16. Christian says:

    Hi Vesa,

    very helpful post, thank you. But it doesn't work for me.

    I've followed the 4 steps, but still getting an ServerException "Argument SiteUrl must be a tenant administration site." (using PowerShell). I already checked the AdministrationSiteType of the particular site, it's "TenantAdministration". Could you point me to the right direction?

    Best regards

    Christian

  17. Vesa Juvonen says:

    Hi Christian,

    that would indicate that the site collection which you connect to as Tenant Admin site is not been congiured properly with the PowerShell on the step 4. This just configures any site collection to act as "tenant admin" site. By default the code is assuming that you have set the root site collection in the used site collection as the "Tenant admin" site, but you can modify the code as needed.

  18. Shriram says:

    Hi Vesa,

    Here i am again!

    While creating a site collection onPremise, I am getting exception "<nativehr>0x80070057</nativehr><nativestack></nativestack>"  at line "adminContext.ExecuteQuery();". I followed steps as mentioned by you and now trying to create a site collection from a console app. The mock admin site is having the same site collection administrator which is used in console app for authentication. And the same user is present in the Farm admins group.

    One thing i would like to highlight is that Installation of April CU 2014 was successful on my farm but somehow Product Configuration wizard failed at step 8. Could this be a reason for this above exception?

    Thanks in advance!

  19. Tony says:

    Hi Vesa, thanks for the article. I completed all 4 requirements. When debug the app, it fails StringComparer.OrdinalIgnoreCase.Equals(httpContext.Request.HttpMethod, "POST") check in CheckRedirectionStatus method of SharePointContext.cs.

    What could be wrong?

    Thanks!

  20. Vesa Juvonen says:

    Hi Shriram,

    if the wizard execution fails, it actually means that the upgrade was not successful and therefore it might be that some of the needed changes have not been applied. You should get much more details on reason from the upgrade log and from ULS log, which should help you to figure out what is blocking the upgrade.

  21. Vesa Juvonen says:

    Hi Tony,

    unfortunately I have not seen that one before, so can't directly say what's wrong. I'd recommend to adjust the farm diagnostic logging settings for the app related topics to verbose and check the additional details from the ULS log.

  22. Nanddeep Nachan says:

    Hello Vesku,

    In on-premise, Is there any possible harm in using actual Central Administration Site instead of creating a Mock Tenant admin site ?

  23. Vesa Juvonen says:

    Hi Nanddeep,

    I have not actually tested this with the Central Admin site and that was not part of the testing with the normal scenario, since the code being executed should anyway not have access to the farm central administration site. Notice that just by running the provided PS command, you can turn into oob team site as "tenant admin site", so there's really nothing that much to be done for making this work.

  24. Jan Steenbeek says:

    Hi Vesa,

    First off, thanks for a great post. It has enabled us to get a long way in creating a hybrid online and on-premises site collection provisioning solution which we are very excited about!

    We're running into a challenge on-premises with selecting web templates. You can only create sites with a web template that is available on the tenant admin site (*). In an online scenario we can create sites without a template and select a template later. This is also blocked on premises. Do you know the purpose of this limitation and whether it will be resolved in a future update? Or could it just be an oversight?

    (*)

    Dll:

    Microsoft.Online.SharePoint.Dedicated.TenantAdmin.dll

    The method:

    Tenant.CreateSite(SiteCreationProperties)

    Calls the following validation method:

    Tenant.ValidateSiteCreationProperties(SiteCreationProperties)

    Which loops through the available web templates on the tenant administration site and throws an exception if the template specified in 'SiteCreationProperties' is not in there.

  25. Vesa Juvonen says:

    Hi Jan,

    good finding on not being able to provision site collection without template like in the cloud. I had not tested that one with on-premises. I've also added note on the used template that it has to be located in the tenant admin site to be able to use that also for newly provisioned sites.

    In general thinking of the implementation has been that the sites are not provisioned using web template in first place, since web templates have maintenance challenge. This is something which is more evident in the cloud than in on-premises, but the challenge is that web templates are not automatically updated when there's improvements introduced to oob templates. In cloud we now release new versions in every second week and some of the improvements are pointed directly to the oob sites. If you have created a web template, you'd have to update that as well, to keep up with the changes, which is not optimal and causes additional costs.

    In general this can be addressed by using CSOM for the other operations as well and essentially dropping the web template usage completely. This way you'd always start from the oob template, like team site, but then based on the user selection or requirements, you'd run the additional code to apply the needed settings like content types, branding, libraries etc. Using this pattern you always get the latest and greatest from the oob sites, but then you modify the structure based on the needs. This model does not however work as end user drive process if the users need to "save sites as templates".

    I do however understand the challenge what you've now faced 100% and will get the feedback directly to the engineering. I would not however expect fast changes (or any) on this process since the full CSOM pattern does work… just being honest on setting the right expectations.

    Thanks for sharing the finding and the issue for others as well. If you are interested on the full CSOM patterns to replace the web template usage, which is where we are gradually heading, have a look on the examples from the Office 365 Developer Patterns and Practices project. Here's for example solution, which for on-premises and does apply custom branding for the created site collection. It's good example of the full CSOM / remote provisioning model which is more aligned with the roadmap of the product and service than continue using web templates – github.com/…/Provisioning.OnPrem.Async

  26. Jan Steenbeek says:

    Hi Vesa,

    Thanks for the clear and elaborate response!

    The message to move away from custom web templates is very clear, and I'm not proposing to do so, but unless the possibility to create sites without a template is added the work around you mention in your update above will not work.

    Regardless of custom web templates, enabling site creation without templates and then applying them in a separate step would appear to be a good way forward because;

    – The same pattern can be used online

    – AFAIK it is not possible to create a tenant admin site with all OOTB web template types available, so you would have to create multiple tenant sites to facilitate creation of all types of sites.

    The CSOM provisioning model you link to is close to what we are trying to achieve and looks helpful, thanks.

  27. Vesa Juvonen says:

    Hi Jan,

    I did not quite understand what would not work. We do this with numerous enterpsei customers in Office 365, in dedicated and in on-premises. Model is that you always start by creating the target site with oob site definition. In practice it's either oob team site or the publishing site, depending slightly on what's needed as the base line. This is the part which will be done first when the site collection is created, so you don't need to apply empty template.

    On following step you modify the oob site based on the selection. You could have for example 5 different adaptations of team sites, which are differentiated with what web parts are on front page and what libraries exists in the site. Based on the user selection, you create oob team site and then apply different settings for different adaptations.

    This exact same pattern can be used online or in on-premises.

    Sorry if I wasn't clear on the first reply on the model. I have blog post pending to clarify this one as well, but struggling to find time to get things out. The Patterns and Practices project under Office Dev shows how this can be done in practice and contains specific samples for different use cases or scenarios like how to apply branding or how to create content types using CSOM.

  28. Jan Steenbeek says:

    Hi Vesa,

    Sorry if I was not clear on what I think won't work. The pattern you describe above in your update:

    " Update on 15th of August: […] Alternatively if you have deployed custom web template, you will have to also make sure that it’s available for the tenant admin site. "

    This specific pattern would not work on premises because to get the custom template available on the new site collection, you would have add it before the template is applied. Hence, a site collection without a template is required. This applies to custom web templates only, which I know are (informally) deprecated.

    I understand what you say about being able to use the same pattern online as we do on premises when using OOTB site templates. That's what we are going to do. Thanks, and have a great weekend.

  29. Hi Vesa / Jan,

    Would it not be possible as a "hack" to provision the site collection using an OOTB site template, then delete the root web in the site collection, upload and activate the custom site template to the new (now empty) site collection solution gallery, and then finally recreate a new root web using the custom site template?

  30. Actually scratch that last comment / suggestion I made. I've since realized you would need a root web site in order have somewhere to upload the custom site template wsp to.

    Thus the "hack would more likely need to do the following:

    1. Provision the site collection using an OOTB site template

    2. Delete the root web in the site collection

    3. Create a new root web in the site collection and not specify the template for the new root

    4. Upload the custom site template wsp to the site collection solution gallery

    5. Apply the custom site template to the new root web

    6. Activate all the required site collection features required

    Doing all this would likely defeat the purpose of using CSOM to provision the site collection in the first place.

  31. Bhaskar says:

    Hi Vesa,

    I used the all procedure in above code but still i have a error faced which are given below:

    An exception of type 'Microsoft.SharePoint.Client.ServerException' occurred in Microsoft.SharePoint.Client.Runtime.dll but was not handled in user code

    Additional information: SiteCreationProperties.Template

    Parameter name: Specified value is not supported for the SiteCreationProperties.Template parameter.

    After that i am using Microsoft.Online.SharePoint.Client.Tenant 15 version of this dll,but problem is still there

    I am used provider hosted appps in high trust in on premises.

    Please give me the reply,it is very urgent.

    I am stuck due to this problem.

    Thank in advance.

    Regards

    Bhaskar

  32. Gerben Ankone says:

    Hi Bhaskar,

    The problem you are facing has to do with the issues described by Jan and Vesa in the previous comments. This exception is being thrown by the code block Jan posted. It occurs when the provided site template is not set as available on your tenant admin site.

    Depending on your situation, it might be possible for you to add the site template you wish to use to the Subsite Templates in Page Layout and Site Template Settings.

    Regards

    Gerben

  33. Vesa Juvonen says:

    Hi Bhaskar,

    thanks Gerben for adding the comment there already. That seems to be precisely the issue, which means that Bhaskar, you'll need to add the oob team site as supported template for the site collection, which you use as the "tenant admin" site.

    If you follow the code as provided, this is the line where we decide the tenant admin site.

        string rootSiteUrl = hostWebUrl.Substring(0, 8 + hostWebUrl.Substring(8).IndexOf("/"));

    Which by default is the root site of the web application. You'll need to go to that site and ensure that you can create out of the box team site as the sub site there. If it's publishing site, you'll need to add out of the box team site as one of the options from site settings.

  34. Peter says:

    Hello,

    Thanks for this great post.

    However, when executing this, I get an error message saying Self-Service Site Creation is not enabled.

    I would find it very strange if this would be required since it is more related with MySites if I understand correctly.

    Did you run into this issue or what could be a solution?

    Thanks

    Peter

  35. Bhaskar says:

    Hi Vesa,

    Now i am able to create the site collection but unable to apply my custom web template.Have you any idea how to apply my custom template and one more thing i am unable to add people picker in provider hosted apps in high trust.Have you any sample for this.

    Regards

    Bhaskar

  36. Vesa Juvonen says:

    Hi Peter,

    Self service site collection creation has to be also enabled if you use out of the box self service site collection creation with normal site collections. This is not something which is only relevant for My Sites. This is exact same requirement for server side code and actually has been the model since SharePoint 2007.

    No changes here.

  37. Vesa Juvonen says:

    Hi Bhaskar,

    That is actually as designed, since usage of web templates would cause challenges with the evergreen model or with any updates you are applying to the sites. We recommend strongly to use pure CSOM model for applying the needed elements to the sites, like activating features, adding branding or creating content types. This is also demonstrated in the updated example at PnP guidance.

    github.com/…/Provisioning.SiteCol.OnPrem

    You will always have to provision the site using out of the box template, but technically you could actually provision empty site and then upload sandbox solution to the site with needed customizations. This model is not however the most optimal one and will cause maintenance challenges due the changes we perform in the oob sites, which are not automatically included to your web template or to the sandbox solutions.

  38. Diederik Tiemstra says:

    Hi Vesu,

    Great post. During testing we sometimes get a time out exception on the adminContext.ExecuteQuery() method for creating the SiteCollection. We did already try to set the RequestTimeout on the adminContext object to a high value. This does not seem to have any affect on the time-out.

    Do you know which setting may cause the Time-outs?

  39. Vesa Juvonen says:

    Hi Diederik,

    Depending on implementation style, this could be caused also IIS time outs, if the code is executed in synchronous ways. You should however follow the async pattern to avoid that as explained in following blog post – blogs.msdn.com/…/async-site-collection-provisioning-with-app-model-for-on-prem-and-office-365-dedicated.aspx

  40. Mike says:

    GetContextTokenFromRequest returns null, any ideas?

  41. Shahid Mahmood says:

    Hi Vesa,

    Great stuff, I am getting  "An error occurred while processing your request." while executing the code, I have followed all the steps that you mentioned, any clue? Also in your subsite provisioning example you had to create a certificate for the S2S communication, wondering why you did not include it here?

  42. Mike says:

    Shahid, in order to get more details, in the site's web.config turn customErrors to Off then try again.

  43. Shahid Mahmood says:

    Thanks Mike, I have the custom errors=off,  its actually coming from the code behind of the default.aspx, from the line  case

    RedirectionStatus.CanNotRedirect:

                       Response.Write("An error occurred while processing your request.");

                       Response.End();

                       break;

  44. Vijay says:

    Shahid,were you able to resolve this? I am having the same error, is there any configuration step required to run this?

  45. Vesa Juvonen says:

    Hi Mike,

    the fact that you are getting token null exception indicates that the trust model is not properly setup either with ACS or with the high trust scenario using certificates.

  46. Vesa Juvonen says:

    Hi Shahid Mahmood,

    we really cannot include the needed certificate since that is dependent on the environment where you use the app. You can also use the low trust model with on-premises with ACS connection to cloud. This is valid alternative to avoid certificate requirements.

    In practice those exceptions indicate either that the trust does not work in general and I'd start by creating simplest possible solution to verify it… just with title read.

    I'd also encourage to have all technical questions routed to Yammer for letting others in community to help. You can find Patterns and Practices yammer group for this kind of topics from aka.ms/OfficeDevPnPYammer.

  47. Mahesh Chintha says:

    Its really a great post and encouraging for app model. I am able to provision a site using OOB template but Is there a way to create a site without template and apply a custom template later? I have seen the way to apply a custom template using CSOM but I would like to create a site without template using Tenant dll.

    Thank you for your suggestions!

  48. Shahid Mahmood says:

    Thankyou Vesa!

    I was able to work this out using high trust model using certificates and instead of getting the token from ACS. I replaced the line var adminContext = TokenHelper.GetClientContextWithAccessToken(tenantAdminUri.ToString(), token)

    with var adminContext = TokenUtility().GetClientContext(this))

  49. Vesa Juvonen says:

    Hi Mahesh Chintha,

    right now applyig template afterwards for the newly created site is not really possible. We do recommend ALWAYS starting with out of the box template and then applying needed settings and customizations using CSOM rather than falling back to sandbox solutions.

  50. Nanddeep Nachan says:

    Hello Vesa,

    I am using this code to provision SiteCollection in on-premise.

    Using CSOM, I want to create a site collection with "Select template later" option so that I can upload the custom web templates there and create a site based on it.

    I get below error, when I pass null as template value:

    Specified value is not supported for the SiteCreationProperties.Template parameter.

    Could you please guide?

  51. Nick says:

    Hi Vesa,

    We have uploaded a custom template to the tenant admin site via the solution gallery and activated the solution. It is correctly showing as an available template in the admin tenant site.  However, when we try to provision a new site collection based off this template, an error is thrown: File or arguments not valid for site template '{2D97CEFF-6EC3-4723-B2C2-9E8BFA737155}#Test Parameter name: WebTemplate.

    As per your update on the 15th of August where it discusses custom templates, is this no longer applicable?

  52. Vesa Juvonen says:

    Nanddeep,

    that is not unfortunately possible with the CSOM. We do recommend usage of CSOM on top of oob templates. This means that you create site collections using oob site definition, but then you apply the needed changes on top of that, which is pretty much what the site template/web template did as well, but this way the site is fully aligned with the oob sites. This is demonstrated with custom theme in following sample – blogs.msdn.com/…/async-site-collection-provisioning-with-app-model-for-on-prem-and-office-365-dedicated.aspx

    Sorry for the delayed response, completely missed the comments. As a rule of a thumb would suggest to post any questions or comments to Office 365 Developer Patterns and Practices Yammer group at the below address. We have 1500 members in this group with app model interest and it's really active group to assist with any questions.

    aka.ms/officedevpnpyammer

  53. Vesa Juvonen says:

    Hi Nick,

    this blog post and comments are valid for on-premises, but not with sandbox solutions. You would have to have that template deployed using farm solutions to make that visible outside of the tenant admin site. Since it now seems to be deployed using sandbox solution, it is not visible for the code trying to create the site.

    We do not recommend usage of web templates / site templates for site creation process due long term impact on template maintenance. You should rather think about provisioning always based on the oob team site. You can still offer different templates for end users, but those are just tags which will differentiate the provisioning process after the oob site definition has been applied.

    Would also suggest to submit any comments or questions using the Office 365 Dev PnP group at – aka.ms/officedevpnpyammer. We have 1500 active members in this group including MS internals and community members. This way we can answer faster on the questions relate don the app model customizations

  54. Ronald says:

    Hello Vesa,

    In a project I'm working on it was noticed that the quota was not set after changing the StorageMaximumLevel on the SiteProperties from the tenant, I then noticed that this also was not set when creating the site collection with the SiteCreationProperties. As I was reading this, I noticed the answer about quota templates.

    As the StorageMaximumLevel has a getter and a setter, I assumed that this could be used to set the Individual Quota. So when you mention "There’s no CSOM to change or select the used quote template at this time.", does this also count for setting the values for the Individual Quota? If so, why does StorageMaximumLevel have a setter?

    I have seen some samples where the StorageMaximumLevel is set on creation of the site collection, so there are others who think this would work. So I might be missing something and it should work, or it just isn't currently supported.

    I guess that setting a individual quota value (when doing an update) where a specific template is already set, that this could cause an issue. For me this could be resolved by returning an error in these cases. That said, it might also be better to only use predefined quota templates, if you look at it from a administrative perspective.

    I hope you can shed some light on this.

    Regards,

    Ronald

  55. Vesa Juvonen says:

    Hi Ronald,

    just to confirm. Quota setting does not work in on-premises. Those properties does exists in the object just because it was ported directly from the cloud CSOM, but those are ignored. Site collections created using CSOM in on-premises will get default quota for them when they are created. There's also no CSOM API to change the quota after the site  has been created.

    Not the best possible situation, but just where we currently are.

  56. Ronald says:

    Vesa,

    Thanks for confirming this, and for your quick response, for now we implemented a message for the admin people to set these values.

  57. Vimlesh says:

    Vesa,

    We want to create  Site Collection on-prem and functionality needs to be exposed through WCF service/ console application. I am able to create  site collection through Provider hosted app but facing issue while creating site colleciton using console application. I am getting  "The remote server returned an error: (401) Unauthorized." on ExecuteQuery statement.  In case of provider hosted app to work I had to use GetS2SAccessTokenWithWindowsIdentity method instesad of GetAppOnlyAccessToken.

    Note : I have generated different certificate and issuer Id for my console app.

  58. Vesa Juvonen says:

    Hi Vimlesh,

    you can create site collection in on-premises without the need of WCF and just with CSOM as explained in this blog post. As long as you used valid identity with CSOM, there's also no need for any oAuth setup for the S2S stuff.

    If you however do combine app model and WCF end points, you will need to remember that the WCF end points do not use oAuth and token handling by default. You cannot directly add this support to the WCF deployed as farm solution to servers, so you will need to handle identity used for calling that separately. You could just as well used typical credential handling and specific service account like with the CSOM approach.

    If you have any other questions, would suggest to use the Office 365 Developer Patterns and Practices Yammer group where we have multiple Microsoft and community members helping on these kind of questions. You can find the group from aka.ms/OfficeDevPnPYammer

  59. Kunal says:

    Hi vesa,

    Will this work with host header web application

    https://dev.somesite.com

    and custom manage path like "client"  to create a site collection like

    dev.somesite.com/…/newsite

  60. Vesa Juvonen says:

    Hi Kunal,

    for the host header questions, answer is that it works as long as the tenant admin site collection is host header site collection as well. I have not tested custom managed paths, but do not see any reasons why they would not work, if you have defined the managed path in central admin.

    If you have any other questions, would suggest to use the Office 365 Developer Patterns and Practices Yammer group where we have multiple Microsoft and community members helping on these kind of questions. You can find the group from aka.ms/OfficeDevPnPYammer.

  61. David says:

    Hi vesa,

    same problem as Christian above:

    "Argument SiteUrl must be a tenant administration site." The property is set on the site, verified with powershell.

    Someone with the resolution to this problem??

  62. Vesa Juvonen says:

    Hi David,

    that's pretty much all that's into it. I can't really explain what goes wrong in your case, since this does definitely work in general. It could be also related on used CU version.

    If you have any other questions, would suggest to use the Office 365 Developer Patterns and Practices Yammer group where we have multiple Microsoft and community members helping on these kind of questions. You can find the group from aka.ms/OfficeDevPnPYammer.

  63. Rajesh says:

    Hi Vesa,

    We are using app model to create Site collection. We are getting time out exception on the adminContext.ExecuteQuery() method for only two webapp's. I added RequestTimeout property to Timeout.Infinite, But still getting Request Timeout issue. Someone has any resolution for this problem.

  64. Vesa Juvonen says:

    Hi Rajesh,

    most likely you are facing challenges with the 90 seconds CSOM process timeout limit configured by default for the SP2013 web applications. You can find sample script to adjust this timeout from the following location, which should resolve your situation: github.com/…/IncreaseOnePremTimeouts.ps1

    If you have any other questions, would suggest to use the Office 365 Developer Patterns and Practices Yammer group where we have multiple Microsoft and community members helping on these kind of questions and this question has been answered there few times. You can find the group from aka.ms/OfficeDevPnPYammer.

  65. Madhu says:

    Thanks for the article.   I'm able to create the site collection in the same farm where the app is running. I'm using the same app (test1) remote provisioning to create the site in another farm (test2) with the app running in test1 farm.   I'm getting the below error

    Original error: System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.     at Microsoft.Online.SharePoint.Dedicated.TenantAdministration.TenantAdminHelper.DoSecurityValidation()     at Microsoft.Online.SharePoint.Dedicated.TenantAdministration.TenantServerStub.InvokeConstructor(XmlNodeList xmlargs, ProxyContext proxyContext)     at Microsoft.SharePoint.Client.ServerStub.InvokeConstructorWithMonitoredScope(XmlNodeList args, ProxyContext proxyContext)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.InvokeConstructor(String typeId, XmlNodeList xmlargs)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.GetObjectFromObjectPath(XmlElement xe)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.GetObjectFromObjectPathId(String objectPathId)

    … at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessInstantiateObjectPath(XmlElement xe)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessOne(XmlElement xe)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessStatements(XmlNode xe)     at Microsoft.SharePoint.Client.ClientMethodsProcessor.Process()

    Any idea would be of great help!

  66. Vesa Juvonen says:

    Hi Madhu,

    that seems simply a permission issue. Resolution slightly depends on the way you are connecting from the farm 1 to farm 2. If you use account based authentication, you'll need to ensure that used account has sufficient permissions to create new site collections. If you are using oAuth, you'll need to register add-in also to farm 2 and give ClientId and Secret combination sufficient permissions using appinv.aspx page.

    If you have any other questions, would suggest to use the Office 365 Developer Patterns and Practices Yammer group where we have multiple Microsoft and community members helping on these kind of questions and this question has been answered there few times. You can find the group from aka.ms/OfficeDevPnPYammer.

  67. Madhu says:

    Hi Vesa,  Thanks for the information! I'm doing this PoC for the first time in multi-farm environment. I'm using S2SGetS2SAccessTokenWithWindowsIdentity apponly authentication.  While debugging, farm2 returns the realm as null. Also I'm not able to add the AppPrincipals in the farm2. Should there be a AppCatalog configured with a appdomain for using in multi farm.  Should the App website be available in both the farms?

  68. Vesa Juvonen says:

    Hi Madhu,

    You will need to configure app domain and app catalog in each of the farms where you are planning to use SharePoint add-ins/apps. When add-in is registered, it's only registered for the farm where you executed the operation.

  69. Ganesh says:

    Please help me – Getting exception "Argument SiteUrl must be a tenant administration site."

  70. Bjørn says:

    Another not usable solution.

    This requires that the root site collection has been created.

    Let's say i have one path based root site collection:

    https:\root.dev.local

    If i mark this site collection as a administration site, i can create https:\root.dev.localsitessite1

    I cannot create a root HNSC with this site collection:

    Like https:\myhnsc.dev.local

    If i create this HSNC site collection manually and mark it as a administration site i can create

    https:\myhnsc.dev.local/sites/site2

    If i create a stand alone administration site with template TENANTADMIN#0  named

    administration.dev.local

    I cannot use this to create site collections like:

    https:\myhnsc.dev.local/sites/site2

    I can only create sitecollections below the administration site like:

    administration.dev.local/…/site3  ??

    So basically you have to tag every root hnsc as a administration site. What a mess..

    And most important, i cannot decide which content database the site collection ends up in ??

    Please ? Who is writing this API and decides, humm content database, no i will skip this option ?

    And the option to enumerate site collections is also not available ?? Method not supported if you call it.

    So basically you have to use search rest apI to get a list of site collections.

    So another crappy solution that doesn't really work in real life.

    This is another CORE functionality that is missing, get it fixed so it works properly.

    I can't believe why core functionality isn't in place, it's not a new API, it's 6 years old!

    I really want to use the client api 100% but it's still CRAP.

    I always have to create a  wcf service that extends the client api, this should not be necessary in 2016.

    Still no listitem history, no system update and a crap system for creating site collections.

  71. Vesa Juvonen says:

    Hi Bjørn,

    thx for your input and feedback. There really valuable input in this comment, which would suggest to get formalized to UserVoice entries, so that we can actually address these API gaps using official channels. APIs are absolutely not perfect and constructive community feedback and suggestions on needed APIs is absolutely welcome. Posting those however to blog posts as comments, if not really the right way to get things changed – since those are anonymous comments without possibility to have open and transparent discussion on your business requirements.

  72. Hi Vesa,

    Thanks for a great article. We have managed to get this solution to work if we set a context that isn’t AppOnly. As soon we try to use the code above to get the context it fails on the row: TokenHelper.GetAppOnlyAccessToken(…) With the following error: {“The remote server returned an error: (400) Bad Request. – {\”error\”:\”invalid_request\”,\”error_description\”:\”AADSTS90002: No service namespace named ‘230e810f-4633-4703-9dbf-57f3520524e4’ was found in the data store.\\r\\nTrace ID: 095f7adb-2cc0-430c-b0e3-3cf29f4ae472\\r\\nCorrelation ID: cdf1fc96-e8ab-4560-9f93-1ba409a8c345\\r\\nTimestamp: 2016-04-05 18:41:55Z\”,\”error_codes\”:[90002],\”timestamp\”:\”2016-04-05 18:41:55Z\”,\”trace_id\”:\”095f7adb-2cc0-430c-b0e3-3cf29f4ae472\”,\”correlation_id\”:\”cdf1fc96-e8ab-4560-9f93-1ba409a8c345\”}”}
    Any ideas?

    Regards, Fredrik

    1. Jose Carlos Pastor says:

      Hi Fredrik,

      I am having the same issue when using this code on SP2016 on-premise. Any idea? Am I missing something?

      Thank you!