Faceted Search 3.0 – implementation of Search Suggestions

  • Article

Background

MOSS Faceted Search provides search refinement to the SharePoint search. In September 2007 I’ve published 1.0 release as an open source on CodePlex. Since then the solution was successfully implemented by small businesses and Fortune 100 corporations in the US and by many companies around the globe.

The latest release of the Faceted Search on CodePlex just came in Version 3.0 Beta. Along with numerous enhancements this release offers:

o Full support of advanced search

o Support of range searches

o Built-in refinement of DateTime and Numeric facets

o A new 3-column layout for publishing

o A new Search Hint web part

In this post I will show the key design decisions and implementation of the Search Hint web part.

image

Introduction

Search Hint feature also known as search suggestions, popular searches, historical searches etc. by now is one of the must have features for any search engine. It allows the user to preview the most popular searches as the one type in the search criteria. Google, Live, Yahoo in one way or another are offering this functionality. The quest to have such done for SharePoint is still ongoing.

The fundamental challenges for the SharePoint implementation are:

  1. where to get past searches from along with their counters
  2. what to do with the MOSS out-of-the-box search box

 

Design

Where to get the data

In December 2008 Microsoft published SharePoint Products and Technologies Protocol Documents. The significance of this event is yet to be fully measured. As for the search suggestions, the [MS-SQLPQ]: Search Service Database Query Protocol Specification outlined the backend schema used by MOSS for storing the usage data. The Search Hint web part uses exactly that, MOSS natural data to provide search suggestions, using very own MOSS SQL artifacts.

As far as Faceted Search uses Enterprise Library for caching, logging and exception handling, the same design is in place for getting usage data from MOSS.

The Search Hint web part doesn’t replace MOSS Search Box. It is designed to be an invisible droplet. When dropped onto the page it is instructed to look around and detect the MOSS Search Box, assess its position and seamlessly attach the search suggestions to it.

Implementation

SQL query

The web part goes to the MOSS Shared Services database using SQL connection defined in the web.config. The T-SQL doesn’t have any direct clause STARTSWITH, so I build the dynamic query using following logic:

SELECT TOP {0} queryString, scope, numScopes, numQueries, FROM ... WHERE LEFT([queryString], ?) = ? ORDER BY ?

Parameters allow managing the number of records in 1 transaction, the match criteria along with the number of characters to match, and the sort order.

There are 2 ways in which Search Box exist on the page.

  1. It can appear as a web part in one of the web part zones, e.g. on the search results page.

private static WebPart FindWebPartByType<T>(Page page)

{

    WebPartManager wpm = WebPartManager.GetCurrentWebPartManager(page);

    if (wpm == null) return null;

    WebPartCollection wpc = wpm.WebParts;

    foreach (WebPart wp in wpc)

    {

        if (wp is T) return wp;

    }

    return null;

}
  1. The search box can be used as ASP.NET control outside web part zone, e.g. in the header of the .master pages.

private static Control FindControlByType<T>(Control control)

{

    Control result = null;

    foreach (Control ctrl in control.Controls)

    {

        if (ctrl is T)

            return ctrl;

        else

        {

            if (ctrl.HasControls())

            {

                result = FindControlByType<T>(ctrl);

                if (result != null)

                    return result;

            }

            continue;

        }

    }

    return result;

}

In either case the finder loops through all the artifacts comparing their types to Microsoft.SharePoint.Portal.WebControls.SearchBoxEx . When found, the web part extracts its ClientID and builds all the positioning in the HTML DOM.

After the Search Box found most of the action occurs on the client side. The control coordinates are extracted using getBoundingClientRect() method and the Search Hint panel is aligned to the search box.

Performance considerations

This functionality can be resource intensive. As such, the 1st thing that was implemented is asynchronous processing of usage data. Implemented, as all over in Faceted Search, using ICallbackEventHandler interface.

Additionally, Search Hint tracks actual keywords while being typed by the user. If previously extracted usage data cover new keyword, no extra call to the server performed. Contrary, the search suggestions are extrapolated from the previously received data feed.

Imagine user searches for “sharepoint” and types “share”. The Search Hint requests all the matching to “share” previous searches, builds resulting table of them and display the polished outcome back to the user. User can choose one of the offered suggestions of course. Say the use keeps typing and produces “sharepo”. The Search Hint doesn’t interrogate the server again because “sharepo” is a subset to “share”. Instead, a very fast filtering of previous data feed occurs and the refined search suggestions are displayed.

Security notion

Getting data from the SQL server requires read permission on the tables. The Search Hint uses SPSecurity.RunWithElevatedPrivileges() wrapper which implies running the process under identity of the current app pool account. In tighten medium- or large- farm deployments, the account in question should be granted DATAREADER privileges to MSSQLog… tables or Shared Services database as a whole.

 

Small things

User accessibility and interface friendliness are frequently overlooked in programming. Here are few nice-to-have things that I’ve implemented in the Search Hint.

Basic settings

The web part allows defining the number of search suggestions to display as well as the sort order. The latter can be alphabetical or by frequency of use. Furthermore, that number (hits) will be shown as a tooltip, should the user hold the mouse on the suggestion.

image

Timeouts and startup

Firing client events on every client click can be exhausting and often annoying. The Search Hint offered a set of timeouts and settings that in fact can be edited directly in FacetedSearch.js library.

var delayKeyWait = 500; - (ms) how long to wait for the next key stroke

var delayHintRetire = 20000; - (ms) how long to keep hint display if no user action

var minKeywordLength = 3; - the minimal number of characters to fire off the search suggestions

Styling

The web part allows tuning of CSS styles for a regular, mouse-over and clicked search suggestions. The interface is consistent with other Faceted Search web parts.

image

Operations

The Search Hint supports both mouse and keyboard interactions. For example, after the user typed a keyword in the Search Box and search suggestions displayed, clicking arrow down/up on a keyboard will scroll through the suggestions. Naturally, the scroll will loop:)

 

Conclusion

More cool features are coming. Stand by

 

About the author

Leonid Lyublinski is a Microsoft consultant leading SharePoint Architecture and Development projects at our biggest clients. He is an author of several CodePlex solutions, a speaker at internal, public and international conferences.