Control Search Indexing (Crawling) Within a Page with Noindex


SharePoint provides the ability to control search indexing (a.k.a. “crawling”) within a page by using a div class named “noindex”.  This means you can flag some content on a page that should not be indexed for search results, while other content on the page is still indexed.  The typical scenario that requires this is common footer text.  Suppose you edit your master page to contain some standard text, including your company name and a link to your privacy policy.  Now, search results for your company or privacy policy will include every page in the site because this text is there.  What you really want is just pages that contain unique content about those keywords.

SharePoint 2010’s iFilter excludes content inside of a <div class="noindex"> tag.  Note that each div nested in that one is re-evaluated for the noindex class, so nested divs may not work as you expect.  The product team is considering changing this so that all nested divs are excluded.

<div class="companyFooterStyle noindex">
      Copyright 2010 © Company Name | <a href="/sitepages/PrivacyPolicy.aspx">Privacy Policy</a>
</div>

Now, when you search the portal for “Company Name” or “Privacy” you won’t get every page in the portal showing up in your results.

Note the ability to use other css classes along with noindex, to get visual styles as well.  I haven’t tested the noindex class on tags other than div.

 

Technorati Tags: ,,

Comments (3)

  1. Hi Mark,

    This is a great improvement that i did not se in MSSE2008, i guess this also applies to Search Server express 2010?

    Will the crawler still follow links within the excluded content? For example in the scenario when you would like to exclude the menu from appearing in the search results, but you still want the links to be further crawled in the hierarchy.

    Finally, i cant seem to find any documentation regarding this, do you know of any?

    Regards

    Daniel Funseth

    Sweden

  2. Jan Siepmann says:

    Hi, I tried this, but the left nav keywords still remain in the index.

    Is there any futher documentation on this topic, I can't find it…

    tia

    Jan

  3. Bob Edwards says:

    Hi, I tried this method to exclude a menu in our master page, but it does not seem to work.

    We are using Windows 2008 Server R2 and Sharepoint 2010 RTM.  Is there an updated iFilter that we need to install?

    This would be a great find if we can get this to work.  Any help would be appreciated.

    Thanks,

    Bob