Generate SharePoint 2010 Sitemap with Windows PowerShell

#Update: This script has been submitted to SharePoint category in TechNet Script Center Repository. 

If you are working on SEO project for your internet facing SharePoint site, one of the top thing would be sitemap generation. There’re quite a few sitemap generation tools on the market, including the sitemap tool in SEO Toolkit for IIS. But do you know you can achieve this just by Windows PowerShell script? It is quite flexible, and easy to customize and automate.

So here is something you can leverage….

New-SPSiteMap Script

 function New-SPSiteMap
{
    param($SavePath="C:\inetpub\wwwroot\wss\VirtualDirectories\80\SiteMap.xml", $Url="https://sharepoint")
    $web=Get-SPWeb $url
    $list = $web.Lists | ForEach-Object -Process {$_.Items}| ForEach-Object -Process {$_.url.Replace(" ","%20")}

     #excludes directories you don’t want in sitemap. you can put multiple lines here:

     $list=  $list | ? {$_ -notmatch "_catalogs"} 

     $list=  $list | ? {$_ -notmatch "Reporting%20Templates"} 

     $list=  $list | ? {$_ -notmatch "Reporting%20Metadata"} 

     $list | New-Xml -RootTag urlset -ItemTag url -ChildItems loc -SavePath $SavePath
}

function New-Xml
{
  
    param($RootTag="urlset",$ItemTag="url", $ChildItems="*", $SavePath="C:\SiteMap.xml")
    Begin {
        $xml="<?xml version=""1.0"" encoding=""UTF-8""?>
        <urlset xmlns=""https://www.sitemaps.org/schemas/sitemap/0.9"">"
        }
    Process {
        $xml += " <$ItemTag>"
        foreach ($child in $_){
        $Name = $child
        $xml += " <$ChildItems>$url/$child</$ChildItems>"
        }
        $xml += " </$ItemTag>"
        }
    End {
        $xml += "</$RootTag>"
        $xmltext=[xml]$xml
        $xmltext.Save($SavePath)
    }
} 

You can download the script here. This script does have some limitations – for example, I didn’t use http encode for the urls so if you have urls in Chinese, they may not be able to be accessed. But certainly one of the best practice of SEO is to avoid using these languages in the url which may cause such encoding. I didn’t implement <lastmod> either.

Usage: New-SPSiteMap –Url [sitename] –SavePath [yoursiteroot\sitemap.xml]

For example, I want to create a sitemap for a SharePoint demo site, so I first need to load the functions into my SharePoint Management Shell, then do the following:

New-SPSiteMap –Url https://www.sharepointisawesome.com –SavePath C:\inetpub\wwwroot\wss\VirtualDirectories\80\sitemap.xml

Robots.txt Customization

The next step is to create a robots.txt with the following content, replace yoursitename with domain name:

Sitemap: https://yoursitename/sitemap.xml

This is to make sure search engines can pick up your sitemap file directly when they read robots.txt. Save it to the same root directory of your SharePoint site.  For example: C:\inetpub\wwwroot\wss\VirtualDirectories\80\

Do a iisreset to ensure the new sitemap.xml and robots.txt can be loaded.  Done!

Search Engine Submission

You can wait for search engines to read your robots.txt and sitemap, or directly submit the sitemap to them.

Example: Submit a new site to Bing.com:

https://www.bing.com/webmaster/WebmasterAddSitesPage.aspx

Fill in web address and sitemap address, then follow the validation instructions.

Automate Sitemap Generation Process

If your site structure is updated frequently, you can also schedule the script to run everyday. This can be achieved by task scheduler. I will not go into the detail here, but don’t forget to use Add-PSSnapin Microsoft.SharePoint.PowerShell to load the snapin before you load the functions.

Jie.