Better Best Bets with Lists in SharePoint 2013

Editor’s note: The following post was written by SharePoint MVP Mikael Svenson

Best Bets or Promoted results is a great way to give end-users the answer to their search query without the additional click to open a page or document to locate the precise information they were looking for. Bing and other search engines already have many examples of this if you for instance look for weather information or currency conversion.

 

Out of the box approach

Out of the box in SharePoint Online/2013 you can set up this type of functionality using Query Rules. But, there are some hitches to the default experience provided by SharePoint.

First of all, the UI is not very user friendly (unless you’re a search expert). Each best bet or promoted result, which is what it is called in SharePoint, need a separate query rule with associated trigger terms. And for each query rule you have to specify the information about the promoted result(s) itself for that rule. All in all it takes a while to get used to, set up and maintain.

Secondly you can only trigger on exact terms/phrases or the start/end of the queries, not getting partial matches or terms spread apart. For on-premises you also have the option to write regular expressions, but now you are moving away from your regular search keyword manager in hurry.

Thirdly you won’t get any lemmatization or stemming on your trigger terms. As an example the trigger term red car will match the following text with lemmatization turned on:

  • red car
  • reds car
  • red cars
  • reds cars

Using lists and one Query Rule to Rule Them All!

I’m not going to take full credit for this idea as it was introduced to me by Petter Skodvin-Hvammen. But I have taken it a bit further to get the lemmatization working. The idea is to create a regular SharePoint list using Enterprise Keywords as the trigger term matcher. An easy and familiar way to add new entries, as well as easy to maintain. Then display the best bet hits from this list in a result block at the top of the result page.

Note: In order to display result blocks your users has to be assigned Enterprise licenses.

To set this up you need only one query rule targeting the best bet list. Once it has been set up you never have to maintain it again, and good bye quirky UI. Adding new best bets is as simple as adding a new row to the best bet list. Something most SharePoint user should be familiar with.

One issue with a list like this is that you would have to query only the Enterprise Keyword column, in order not to get hits from matches in the title, description or URL fields of the list items. This is all good, but when executing a property search you get no lemmatization/stemming on your trigger terms.

My post “What makes a SharePoint column searchable?” serves as background material to my refined approach to get lemmatization going without doing a property query or matching on title and description.

By querying against the full-text index and not one property you get the added benefit of partial matches on the trigger terms. In the scenario of the trigger term being red car the best bet block rule would trigger for the below queries as well.

  • red
  • reds
  • car
  • cars
  • (network OR red) AND (test OR car)

If you have multi-word trigger terms, make sure all words are pretty good/unique to avoid unwanted partial matches. Or stick with the out of the box query rule matching with a promoted result instead.

The re-written query I will use at the end will limit results to items where the content type is BestBets, and execute a full-text query on those items.

spcontenttype:bestbets {searchTerms}

What’s needed to set this up is

  1. A Best Bet list hosted on a SharePoint site. Don’t use the Search Center as it’s marked as not to be indexed. A separate site collection or sub-site anywhere outside the Search Center is fine, or even as a sub-site to a Search Center as long as you make sure it’s being indexed in some way.
  2. Editing or mapping of your crawled properties to reduce recall on title, description and URL columns.
  3. Custom Display Template for the Best Bets
  4. Custom Result Type and Result Source to target the Best Bet list
  5. Query Rule with result block targeting the Best Bet list using the custom result source

Step 1 – Creating a site, columns, content type and the list

Start by creating a new site or sub-site based on the Team Site template as there is no blank site template in the SharePoint UI.

On your newly created site, add site columns as defined below and add them to a new content type named BestBets. The name in parenthesis are the internal column names.

  • Title (bbTitle – Single line of text *Required)
  • Description (bbDescription – Multiple lines of text)
  • URL (bbLink – Single line of text)
  • Start Date (bbStartDate – Date & Time)
  • End Date (bbEndDate – Date & Time)
  • Enterprise Keywords (enable on the list)

You might be curious as to why I’m not using a HyperLink column for the URL, and the reason is that a URL column will always be included in the recall, even though mapped to an unsearchable managed property (bug anyone?).

You can also add more columns to suit your management and display of information needs.

 

Next, hide the default Title column so you don’t have to deal with triggering on that column as you have no control over where it’s mapped in regards to content recall.

The reason for using site columns is to get automatic managed properties for retrieval (as mentioned in “What makes a SharePoint column searchable?”). The automatically generated managed properties will be used in the Display Template.

Once set up, your content type should look similar to the below image.

 

Create a new custom list on your site, turn on management of content of content types (Settings->Advanced Settings->Allow management of content types) and add the BestBets content type to your list and make it the default content type for the list. Make sure everyone has read access to this list. If not they won’t get any best bets. You may also utilize the security of list items to limit who get’s which best bet. An added bonus.

 

Step 2 – Add content and tune the crawled property settings

Add one best bet to your best bets list and kick off or wait for a crawl to pick up your data and create the crawled and managed properties needed.

 

If you issue the query: spcontenttype:bestbets rose, after the initial indexing you will get a hit on the word roses from the description column, which is undesirable, as you only want matches in the Enterprise Keywords column.

This is where it gets tricky. If you are on-premises you can edit the crawled properties in the search schema on the SSA and turn off the option to include them in the full-text index. For SharePoint Online you have to take a different approach as you cannot edit crawled properties.

To ensure you don’t get recall from unwanted text, create a new managed property which is marked as not searchable, and map all the textual ows_bbInternalName crawled properties to it.

As I’m using SPO for my prototyping I have creating a new managed property at the site collection root of my search center called BestBetsNoRecall, and mapped the following crawled properties to it:

  • ows_bbDescription
  • ows_bbTitle
  • ows_bbUrl

 

The beauty and perhaps a side effect of this Prevent Recall type of managed property is that even if it’s done locally at the site collection/site, you won’t get recall for those columns if you search from another site/site collection either. You have effectively made that column non-searchable.

Troubleshooting: If you cannot get the NoRecall property to work, try to create the property and mappings at the SSA/tenant level instead.

If you want to add support for start date and end date of the best bets, map the crawled property ows_bbStartDate to the managed property RefinableDate00 and ows_bbEndDate to RefinableDate01.

Once you have completed the property mappings, go to advanced settings for your list and click the Reindex List button to make sure the best bets are re-processed on the next crawl.

Step 3- Create a Display Template

The Display Template are stored in the master page gallery, and the result type and result source has to be created at either the Search Center level or globally on the SSA/tenant in order to make them available. Basically you have to create search settings at the same level or at a parent level to where you are using them.

For the display template create a copy of the file _catalogs/masterpage/Display Templates/Search/Item_BestBet.html and name it Item_BetterBestBet.html.

Open the copied file in a text editor and edit the <title> tag to read Better Best Bet Item. Next add the following to the header of the display template:

<mso:ManagedPropertyMapping msdt:dt="string">'bbTitleOWSTEXT':'bbTitleOWSTEXT','bbDescriptionOWSMTXT':'bbDescriptionOWSMTXT','bbUrlOWSTEX':'bbUrlOWSTEXT'

</mso:ManagedPropertyMapping>

Below the line $setResultItem(itemId, ctx.CurrentItem); insert the following lines to quickly map the properties to the default ones and get the template working.

ctx.CurrentItem.Title =  $getItemValue(ctx,"bbTitleOWSTEXT").value;

ctx.CurrentItem.Description =  $getItemValue(ctx,"bbDescriptionOWSMTXT").value;

ctx.CurrentItem.Url =  $getItemValue(ctx,"bbUrlOWSTEXT").value;   

There are many ways to modify a display template to get this working, but this is one quick way to get it up and running. See SharePoint 2013 Design Manager display templates for more information on display templates. You might also want to create a custom control template to customize or remove the border around the best bet items.

Note: Make sure you publish your display template once complete to make it accessible to all users.

Step 4 – Create Result Type and Result Source to target the Best Bet list

Navigate to Manage Result Types on your Search Center (Site Settings->Search Result Types - site collection) and add a new result type with settings like depcited below. For ContentType you must use Contains any of… and not Equals, as this property is a but quirky and SPContentType is not accessible in the dropdown. A better option might be to use the ContentTypeId.

 

Next create a new Result Source (Site Settings->Search Result Sources) with the following properties:

  • Name: Better Best Bets
  • Protocol: Local SharePoint
  • Type: SharePoint Search Results
  • Query Transformation: {?{searchTerms} SPContentType:BestBets ((RefinableDate00<=today AND RefinableDate01>=today) OR (RefinableDate00<>"this year" AND RefinableDate01<>"this year")) }

I want to point out the RefinableDate<>”this year” parts of the query, which will include any best bet which is NOT tagged with a  start and end date. It’s a workaround to include results which don’t have a value.

Step 5 – Create a Query Rule to serve up Best Bets

Now for the final piece of the puzzle. Add a new Query Rule (Site Settings->Search Query Rules->Context=Local SharePoint )

  • Rule name: Better Best Bets
  • Query Conditions: Remove any conditions
  • Add a Result Block
    • Block title: Best Bets for “{subjectTerms}”
    • Query->Select this Source-> Better Best Bets
    • Items: 5
    • Settings->This block is always shown above core results
  • Change ranked results by changing the query: {searchTerms} -spcontenttype:bestbets

What the rule does is include up to five best bet results from the Best Bet list at the top of the results, and also exclude those items from the regular results themselves.

The End Result and next steps

Executing a search for red cars you now get a best bet at the top, even lemmatized.

 

Next steps would be to improve on the Control and Display Template to make it more visual appealing, and you could for example incorporate an image link or other actions.

I’m not saying it’s easy to set up if this is the first time you work with SharePoint 2013 search settings, but once set up your keyword managed for Best Bets will be eternally happy for providing him or her with a UI they actually can manage.

 

About the author

Mikael Svenson is a principal Consultant at Puzzlepart where he develops SharePoint Business Apps and consults on SharePoint in general. Mikael is a search enthusiast at heart having authored "Working with Microsoft FAST Search Server 2010 for SharePoint". A four time SharePoint Server MVP, Mikael puts his local community efforts into being a board member of both the Norwegian SharePoint community and SharePoint Saturday Oslo. In addition to organizing, he also speaks at conferences, events and user groups. Mikael has a blog at techmikael.blogspot.com where he mostly blogs about SharePoint and search, but you can find other nuggets there as well. You can follow @mikaelsvenson on Twitter or check out some forum goodness over at TechNet (https://social.technet.microsoft.com/profile/mikael%20svenson/)

About MVP Monday

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager, formerly known as MVP Lead, for Messaging and Collaboration (Exchange, Lync, Office 365 and SharePoint) and Microsoft Dynamics in the US. She began her career at Microsoft as an Exchange Support Engineer and has been working with the technical community in some capacity for almost a decade. In her spare time she enjoys going to the gym, shopping for handbags, watching period and fantasy dramas, and spending time with her children and miniature Dachshund. Melissa lives in North Carolina and works out of the Microsoft Charlotte office.