Search scopes in FAST Search for SharePoint (Part 2)


 

 

Introduction

In my previous blog post about search scopes I described how to manage scopes with FAST Search for SharePoint (FAST Search). In this blog post I’ll try to describe how to programmatically set a filter on a scope, how to fetch the scope rules for a scope and outline how you can convert it’s scope rules into FAST Query Language (FQL) filters.

Setup development environment

See my blog post about accessing FAST admin features for how to set up the development environment, for this exercise it’s similar, except you don’t need the Microsoft.SharePoint.Search.Extended.Administration.dll assembly

Implementation

Updating the scope filter value

First I’ll show how you can fetch a site scope and a shared scope and modify the filter parameter, it’s pretty straight forward. For both site scopes and shared scopes you need to fetch a ScopesManager, the only difference is how you fetch it. When you have the ScopesManager you can fetch a named scope from it. This code assumes you have a site called ‘fast’ with the scope “MySimpleSiteScope” and a shared scope “MySimpleSharedScope”

using System;

using Microsoft.Office.Server.Search.Administration;

using Microsoft.SharePoint;

 

namespace FASTScopeFilters

{

    class Program

    {

        static void Main(string[] args)

        {

            try

            {

                UpdateSiteScope();

                UpdateSharedScope();

            }

            catch (Exception e)

            {

                Console.Out.WriteLine("Exception caught: " + e);

                Environment.Exit(1);

            }

        }

 

        private static void UpdateSiteScope()

        {

            const string siteCollectionUrl = "http://localhost/fast/";

            const string scopeName = "MySimpleSiteScope";

 

            using (var site = new SPSite(siteCollectionUrl))

            {

                var scopesManager = new RemoteScopes(SPServiceContext.GetContext(site));

 

                var scope = scopesManager.GetScope(new Uri(site.Url), scopeName);

                Console.Out.WriteLine("Site scope filter value before update: {0}", scope.Filter);

                scope.Filter = "title:site";

                scope.Update();

            }

        }

 

        private static void UpdateSharedScope()

        {

            const string scopeName = "MySimpleSharedScope";

 

            foreach (SearchServiceApplication ssa in SearchService.Service.SearchApplications)

            {

                ScopesManager scopesManager = new Scopes(ssa);

                var scope = scopesManager.GetSharedScope(scopeName);

                Console.Out.WriteLine("Shared scope filter value before update: {0}", scope.Filter);

                scope.Filter = "title:shared";

                scope.Update();

            }

        }

    }

}

 

Converting a scope’s rules into a FQL filter string

Say you already have an existing Search Server scope defined, and you want the rules of this scope converted into an FQL filter string that matches this scope. If we were to do that in the UpdateSiteScope() method above, the code would have to be updated to look something like this:

private static void UpdateSiteScope()

{

    const string siteCollectionUrl = "http://localhost/fast/";

    const string scopeName = "MySimpleSiteScope";

 

    using (var site = new SPSite(siteCollectionUrl))

    {

        var scopesManager = new RemoteScopes(SPServiceContext.GetContext(site));

 

        var scope = scopesManager.GetScope(new Uri(site.Url), scopeName);

        Console.Out.WriteLine("Site scope filter value before update: {0}", scope.Filter);

 

        var searchApp = (SearchServiceApplicationProxy)

            SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(site));

 

        // status code != 0 means the scope doesn't exist

        int statusCode;

        var rulesForScope = searchApp.GetRulesInfo(scope.ID, out statusCode);

 

        var scopeHelper = new ScopeHelper(rulesForScope);

 

        scope.Filter = scopeHelper.GenerateFastScopeFilter();

        scope.Update();

    }

}

 

I’ll leave it as an exercise for the reader to figure out the code in ScopeHelper to generate the FQL from scope rules, but it’s not very difficult given you know how the different scope rules map to FQL terms, which I’ll describe how to do next.

Converting a scope’s rules into an FQL filter

A search scope rule can have three behaviors; Include, Exclude or Require.  Include represents logical "OR", Require represents logical "AND" while Exclude represents logical "NOT". When converting scope rules into FQL statements all rules are converted to property restriction queries based on rule type.

Property Query rules

Property query scope rules are created the same way you would write a property restriction in your query.

The Include property/value pair includeprop/includeval is translated into the FQL statement:

includeprop:”includeval”

Whereas the Exclude property/value pair excludeprop/excludeval is translated into the FQL not statement:

not(excludeprop:”excludeval”)

A combination of an Include/Require and an Exclude rule is translated into the following FQL:

andnot(includeprop:”includeval”,  excludeprop:”excludeval” [,excludeprop2:”excludeval2”, …])

Web Address rules

Web address scope rules are converted into property restrictions using the managed properties "path" and "domain", depending on the type of the rule.

For Folder rules the property used in the property restriction is the managed property “path”.

An UrlScopeRule that has a folder rule type will look like this:

path:starts-with(“http://mysharepoint/sites/mysite/myfolder/)

For Domain and Hostname rules the property used in the property restriction is the managed property “domain”, but with a little difference in usage depending on rule type.

An UrlScopeRule that has a domain rule type will look like this:

domain:”mydomain”

An UrlScopeRule that has a hostname rule type will look like this:

domain:starts-with(“myhost”)

The Include/Exclude/Require statements are similar as for Property Query rules.

Content Source rules

In a FAST Search for SharePoint setup you will have two search service applications (SSA): one for content and one for query. Since you set up content sources on the content SSA and scope rules on the query SSA, there is unfortunately no information in the query SSA about the content sources in the content SSA. To achieve content source rule filters the approach is to use property query scope rules instead, where the managed property is “ContentSource” and the value is the name of the content source:

contentsource:"local sharepoint sites"

All Content rules

“All Content” rules are not needed in FAST Search for SharePoint and can be ignored. But do note that you can attach a filter also to an “All Sites” type of scope should you want to.

Putting it all together

A scope sometimes consist of more than one rule, which means your FQL filter will grow more complex the more rules you have. In the following section I’ll try to outline how rules are combined into FQL filters below, reading the FQL chapter here is recommended although the examples below should be understandable without it.

Single rule examples

Property Query Rule

Rule Name

Rule1

Filter Behavior

Include

Property

contentclass

Value

I1

FQL Filter

contentclass:”I1”

 

Web Address Rule

Rule Name

Rule1

Filter Behavior

Include

Type

Folder

Value

http://sitecollection/

FQL Filter

path:starts-with("http://sitecollection/")

Domain rule

Rule Name

Rule1

Filter Behavior

Include

Type

Domain

Value

domain.com

FQL Filter

domain:"domain.com"

 

Hostname rule

Rule Name

Rule1

Filter Behavior

Include

Type

Hostname

Value

hostname

FQL Filter

domain:starts-with("hostname")

 

Only Include Rules

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Include

Include

 

Property

contentclass

contentclass

 

Value

I1

I2

 

FQL Filter

contentclass:”I1”

contentclass:”I2”

or(contentclass:"I1", contentclass:"I2")

 

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Include

Include

 

Type

Folder

Folder

 

Value

http://sitecollection/

http://sitecollection2/

 

FQL Filter

path:starts-with("http://sitecollection/")

path:starts-with("http://sitecollection/")

or(path:starts-with("http://sitecollection/"),

path:starts-with("http://sitecollection2/"))

 

 

Two Include rules are combined into an or() statement.

Only Require Rules

Rule Name

Rule1

Filter Behavior

Require

Property

contentclass

Value

R1

FQL Filter

contentclass:”R1”

 

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Require

Require

 

Property

contentclass

contentclass

 

Value

R1

R2

 

FQL Filter

contentclass:”R1”

contentclass:”R2”

and(contentclass:"I1", contentclass:"I2")

 

Two require rules are combined into an and() statement.

Only Exclude Rules

Rule Name

Rule1

Filter Behavior

Exclude

Property

contentclass

Value

E1

FQL Filter

not(contentclass:”E1”)

 

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Exclude

Exclude

 

Property

contentclass

contentclass

 

Value

E1

E2

 

FQL Filter

contentclass:”E1”

contentclass:”E2”

or(not(contentclass:"I1"), not(contentclass:"I2"))

 

Two or more exclude rules are OR’ed together.

Include and Require Rules

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Include

Require

 

Property

contentclass

contentclass

 

Value

I1

R1

 

FQL Filter

contentclass:”I1”

contentclass:”R1”

and(contentclass:"I1", contentclass:"R1")

 

Combining an Include and a Require rule results in an and() if there is only one include rule.

Rule Name

Rule1

Rule2

Rule3

Combined

Filter Behavior

Include

Include

Require

 

Property

contentclass

contentclass

contentclass

 

Value

I1

I2

R1

 

FQL Filter

contentclass:”I1”

contentclass:”I2”

contentclass:”R1”

and(or(contentclass:"I1", contentclass:”I2”), contentclass:"R1")

 

Two include rules and one require rule results in an and() between the include rules or()’ed together and the require rule.

Include/Require and Exclude rules

Rule Name

Rule1

Rule2

Combined

Filter Behavior

Include

Exclude

 

Property

contentclass

contentclass

 

Value

I1

E1

 

FQL Filter

contentclass:”I1”

contentclass:”E1”

andnot(contentclass:"I1", contentclass:"E1")

 

Rule Name

Rule1

Rule2

Rule3

Combined

Filter Behavior

Include

Exclude

Exclude

 

Property

contentclass

contentclass

contentclass

 

Value

I1

E1

E2

 

FQL Filter

contentclass:”I1”

contentclass:”E1”

contentclass:”E2”

andnot(contentclass:"I1",  contentclass:"E1",  contentclass:”E2”)

 

Rule Name

Rule1

Rule2

Rule3

Rule4

Combined

Filter Behavior

Include

Include

Exclude

Exclude

 

Property

contentclass

contentclass

contentclass

contentclass

 

Value

I1

I2

E1

E2

 

FQL Filter

contentclass:”I1”

contentclass:”I1”

contentclass:”E1”

contentclass:”E2”

andnot(or(contentclass:"I1", contentclass:”I2”),  contentclass:"E1", contentclass:”E2”)

 

Rule Name

Rule1

Rule2

Rule3

Rule4

Combined

Filter Behavior

Require

Require

Exclude

Exclude

 

Property

contentclass

contentclass

contentclass

contentclass

 

Value

R1

R2

E1

E2

 

FQL Filter

contentclass:”R1”

contentclass:”R2”

contentclass:”E1”

contentclass:”E2”

andnot(and(contentclass:"R1", contentclass:”R2”),  contentclass:"E1", contentclass:”E2”)

 

With Include/Require and Exclude, the first statement of and andnot() are the Include/Requires while the rest are the Excludes.

Include, Require and Exclude rules

Rule Name

Rule1

Rule2

Rule3

Combined

Filter Behavior

Include

Require

Exclude

 

Property

contentclass

contentclass

contentclass

 

Value

I1

R1

E1

 

FQL Filter

contentclass:”I1”

contentclass:”R1”

contentclass:”E1”

andnot(and(contentclass:"I1", contentclass:"R1"), contentclass:"E1"))

 

Rule Name

Rule1

Rule2

Rule3

Rule4

Combined

Filter Behavior

Include

Include

Require

Exclude

 

Property

contentclass

Contentclass

contentclass

contentclass

 

Value

I1

I2

R1

E1

 

FQL Filter

contentclass:”I1”

Contentclass:”I2”

contentclass:”R1”

contentclass:”E1”

andnot(and(or(contentclass:"I1", contentclass:”I2”), contentclass:"R1"), contentclass:"E1"))

 

Summary

You can create FQL filters that match existing scope rules programmatically if you already have scopes defined and are migrating to using FAST Search for SharePoint. In case you’re using both SharePoint Search and FAST Search you can keep scope rules and filters in sync with each other. Do note that in case you’re only using FAST Search you don’t have to create any rules for your scopes, all you need to do is define the scope and then set a filter term, either using PowerShell or doing it programmatically.

Skip to main content