Full Text Search Capabilities in Azure AD Graph API (preview)

Advanced warning notice:  The Azure AD Graph team will be shutting off this preview capability.  As of September 19th 2016, this feature will no longer be available.  We'd like to hear from you if you have been using this capability, and what you think about it.  We are currently investigating some options around introducing this type of capability again at a future (unspecified) date.

 

The Azure AD Graph team is pleased to announce the public preview of full text search capabilities. Until now, searching AAD entities was limited to $filter searches and had a number of limitations developers found constraining. With this preview functionality we add more powerful search mechanisms to the AAD Graph.

What this enables

In this section we’ll talk about different scenarios that can be achieved using this new full text search capability, together with some example REST calls.

AAD Graph now has the ability to leverage a full text index to move beyond the limitation of only being able to search for words that appeared at the start of the property text and words which followed immediately after one another.

As an example, if you wanted to discover all the company discussion groups you might do a search for “Contoso discussions” across group name and description. However, using filter searches, you’d miss groups titled “Discussions about Contoso”, “Contoso Social Discussions”, “DL for Contoso Discussions” etc. This is not the case for full text searches where it will search the two tokens “Contoso” and “discussions” across all of the objects in scope.

To perform a full text search, you must be targeting either Users, Contacts, or Groups and you must use the api-version “beta”, then use the $search argument to pass your search text.

GET /contoso.com/users?api-version=beta&$search=search text

Searching across all fields at once

The default for $search will be to search across all indexed fields.  This means it’s no longer necessary to form a query such as:

GET /contoso.com/users?api-version=1.5&$filter=startswith(displayName,'james') or startswith(givenName,'james')

Instead, you can simply make the query:

GET /contoso.com/users?api-version=beta&$search=james

NOTE: These searches are not identical. In this case the full text search would return a user with the name of “John James” as well as “James Doe”

Searching across specified fields

Even though the default for search is to evaluate all fields there are scenarios where it is useful to restrict the fields. This can be accomplished using the “searchable-fields” argument when making the search.  For example you may want to search for all the users named James that are in building 2 but you don’t want to pull up users with 2 in another field:

GET /contoso.com/users?api-version=beta&$search=james+2&searchable-fields=displayName,physicalDeliveryOfficeName 

Ordering by any property

Previously if you wanted to sort by a property that was the only property you could do your filtering with. Another valuable capability that also gets introduced is searching on one field but sorting on another.

To search for all users with James showing up on their user object but ordered by jobTitle, the following search can be performed:

GET /contoso.com/users?api-version=beta&$search=james&$orderby=jobTitle

Of course searchable-fields could be used to restrict the properties the $search is relevant for and it doesn’t have to contain jobTitle.

Paging

Previously filtering, sorting and paging through a single AAD Graph API REST request was not possible. However, paging is now possible in all types of search through the use of the $top and $skip arguments.

For example, you can fetch a page of 10 users containing “James”, ordered by job title:

GET /contoso.com/users?api-version=beta&$search=james&$orderby=jobTitle&$top=10

Then, subsequently use this query to get the next set:

GET /contoso.com/users?api-version=beta&$search=james&$orderby=jobTitle&$top=10&$skip=10

NOTE:   Unlike regular Graph API calls, when using $top with full-text search, an @odata.nextLink will not show up in the response.  For paging, you’ll need to use $skip.

Text-Mode

By default all terms in the search text must be contained in the object, for the object to be returned. However this can be controlled with the “text-mode” parameter. So if you want to simply ensure that any of the search terms match, you can set “text-mode” to “any”.

Using the “any” mode, a search for “John Doe” would return all the John’s and all the Doe’s.

GET /contoso.com/users?api-version=beta&$search=john doe&text-mode=any

More complex queries

+ AND Operator. James+developer would find all the objects with James and Developer.
| OR Operator. James|Karl would find all the objects with James or Karl.
- NOT Operator. James -developer would find all the objects with James which don’t have developer
* Suffix Operator. Dev* would find objects with Dev, Development, Developer, etc.
( ) Precedence operators. Discussions+(Contoso|Fabrikam) will find objects containing “Discussions” and either “Contoso” or “Fabrikam” (or both).
Phrase indicator. Indicates that order should be respected. "Contoso Discussions" won’t find "Contoso Technical Discussions" but would find "DL for Contoso Discussions"

As an example of a more complex search, the following would search for objects which contained James and either developer or the string ‘project manager’ but not Doe:

GET /contoso.com/users?api-version=beta&$search=james+(developer|”project manager”) -doe

Limitations

Read-Write Consistency

This full text search offering is made possible by an index separate from the main data storage. That means that changes made to the directory must replicate to the index. This generally happens in just a few seconds however it can lead to some inconsistencies when doing lots of edits and interactively performing searches expecting the edits you just made to be reflected.

NOTE: This has the consequence of, in certain race conditions, to return objects which used to match the search terms but no longer do. The search results will however always eventually converge with the data store and this behavior should be rare.

Object Limitations

Currently this full text search capability is limited to only Users, Contacts and Groups. Other objects types (Applications, Devices, etc.) may have support added to them depending on community feedback.

Suffix searches

Suffix searches suffer from a few problems:

  • Suffix searches have very poor performance especially when using only a few letters before the suffix operator. Suffix searches using only one or two characters are not recommended.
  • Suffix searches fail when the search text contains word breaking characters. If someone’s email address is James-Doe@contoso.com or James.Doe@contoso.com a search for James.D* or James-D* will not find these objects.

Paging

As noted earlier, unlike regular Graph API calls, when using $top with full-text search, an @odata.nextLink will not show up in the response.  For paging, you’ll need to use $skip.  We’ll look to rectify this in the future.

Upcoming additions

The following additions to search capability are being investigated. We welcome community feedback on which are the most valuable.

Multi-object search support

Currently only one object can be searched at a time. In the future we will provide the ability to search across multiple types of objects.  This will allow you to write one query to give you a true people picking search capability (that does full text search across multiple objects like users and groups, ordering by display name and getting back paged results).

Performance Improvements

Performance improvements for search in general as well as specifically for suffix searching capabilities are being investigate.

Additional object type support

Adding the ability to perform searches for objects other than Users, Contacts and Groups.

The ability to search with a fuzzy edit distance. This takes care of most types of misspellings (e.g. ‘Hames’ finds ‘James’).

The ability to specify a phonetic search so terms that sound alike can be returned (e.g. ‘John’ and ‘Jon’ could be returned with a phonetic search of ‘Jawn’).

Graph client library support

Currently full-text search is only available through the REST API.  This feature would expose full-text search capabilities through the Graph Client Library

Feedback

We'd love to hear what you think about the new full text search capability, any issues you see and any suggestions you might have for improvements or new features and additions.