Outlook 2007 RTM Docs - Notification Based Indexing Support
[This is now documented here: https://msdn.microsoft.com/en-us/library/bb821096.aspx, https://msdn.microsoft.com/en-us/library/bb821097.aspx, https://msdn.microsoft.com/en-us/library/bb821099.aspx, https://msdn.microsoft.com/en-us/library/bb821100.aspx]
(Now that Outlook 2007 is available I'm reposting some of the articles from the Outlook 2007 Beta Documentation series. Some of the articles are virtually unchanged, others are completely different to reflect the differences between Beta and RTM. This article is the updated version of the first half of Outlook 2007 Beta Documentation - Notification Based Indexing Support.)
New flag that identifies stores as using Pusher model when indexing
This flag is set by pusher stores.
Definition
#define STORE_PUSHER_OK ((ULONG) 0x00800000)
If the store provider sets this flag the MAPI PH won’t crawl the store and the store is responsible to push any changes through notifications to the indexer to have messages indexed.
Usage
This flag can be retrieved by getting the property PR_STORE_SUPPORT_MASK from the store.
MAPI URLs
Every time a folder, message or an attachment is to be indexed a unique Unicode URL has to be generated and sent to the indexer. This URL will later be used to identify which object to be indexed in the MAPI PH that runs in the Search Protocol Host. Store providers are responsible for generating these URLs.
MAPI URLs have the following format:
Mapi://SID/StoreDisplayName ($HashNumber)/StoreType/FolderNameA/…/FolderNameN/[EntryIDEncoded[/at=AttachIDEncoded:FileName]]
Parameters
SID
Current user’s SID
HashNumber
DWORD in hex calculated based on the store entry id or the store mapping signature. This value will be stored in the registry and will be used later to identify the store in the MAPI PH.
This number has to be calculated in a way that minimizes collisions between other stores. See below for the algorithm Outlook uses to calculate the hash.
StoreType
Number that identifies the type of the store that contains the object to be indexed. Here are the possible values:
0 – Default store
1 – Delegate store (used for delegate items cached locally)
2 – Public Folders (used for public folder favorites)
Note that if the store is being crawled instead of pushed the value used is none of the above but instead it will be the character X.
FolderNameA/…/FolderNameN
The path from the root of the IPM_SUBTREE to the folder or message. For instance, a message in the “Family” folder under “Inbox” will have Inbox/Family for this parameter.
EntryIDEncoded
MAPI Entry ID for the item encoded as a Unicode string. See below for how it gets encoded. Note that when viewed as text, this encoded entry ID will appear as random Hangul characters or boxes depending on available fonts.
AttachIDEncoded
Attachment ID encoded as a Unicode string.
FileName
Attachment file name as it appears in the message.
Here are some examples of MAPI URLs for a folder, message and attachment respectively:
mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($be19928f)/2/Office
mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($484efb89)/0/Calendar/곯가가가걍걝걌곌겷걢곒갑겛개가검걟곔걙곾걤곂갠가
mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($484efb89)/0/Inbox/곯가가가걍걝걌곌겷걢곒갑겛개가검걟곔걙곾간곷갦가/at=겅걋각가:somefile.txt
Characters that get encoded if they are in the store or folder display name
Here is the table that shows which characters we encode:
% -> %25,
/ -> %2F
\ -> %5C
* -> %2A
? -> %3F
Algorithm to calculate the store hash
Note that if a store has PR_MAPPING_SIGNATURE in the global profile section then that’s what we will hash instead of the store entry id. For all other stores we will hash the Entry ID.
DWORD ComputeStoreHash(ULONG cbStoreEID, LPENTRYID pbStoreEID, LPCWSTR pwzFileName)
{
DWORD dwHash = 0;
ULONG cdw = 0;
DWORD* pdw = NULL;
ULONG cb = 0;
BYTE* pb = NULL;
ULONG i = 0;
// Get the Store Entry ID
// pbStoreEID is a pointer to the Entry ID
// cbStoreEID is the size in bytes of the Entry ID
pdw = (DWORD*)pbStoreEID;
cdw = cbStoreEID / sizeof(DWORD);
for (i = 0; i < cdw; i++)
{
dwHash = (dwHash << 5) + dwHash + *pdw++;
}
pb = (BYTE *)pdw;
cb = cbStoreEID % sizeof(DWORD);
for (i = 0; i < cb; i++)
{
dwHash = (dwHash << 5) + dwHash + *pb++;
}
// You may want to also include the store file name in the hash calculation
// Get store FileName
// pwzFileName is a NULL terminated string with the path and filename of the store
if (pwzFileName)
{
while (*pwzFileName)
{
dwHash = (dwHash << 5) + dwHash + *pwzFileName++;
}
}
// dwHash now contains the hash to be used. It should be written in hex when building the URL.
return dwHash;
}// ComputeStoreHash
Algorithm to encode the Entry ID and the attachment ID
The goal of this algorithm is to generate a compact representation of the Entry ID or attachment ID.
const WORD kwBaseOffset = 0xAC00; // Hangul char range (AC00-D7AF)
LPWSTR EncodeID(ULONG cbEID, LPENTRYID rgbID)
{
ULONG i = 0;
LPWSTR pwzDst = NULL;
LPBYTE pbSrc = NULL;
LPWSTR pwzIDEncoded = NULL;
// rgbID is the item Entry ID or the attachment ID
// cbID is the size in bytes of rgbID
// Allocate memory for pwzIDEncoded
pwzIDEncoded = new WCHAR[cbEID];
if (!pwzIDEncoded) return NULL;
for (i = 0, pbSrc = (LPBYTE)rgbID, pwzDst = pwzIDEncoded;
i < cbEID;
i++, pbSrc++, pwzDst++)
{
*pwzDst = (WCHAR) (*pbSrc + kwBaseOffset);
}
// Ensure NULL terminated
*pwzDst = L'\0';
// pwzIDEncoded now contains the entry ID encoded.
return pwzIDEncoded;
}// EncodeID
New notification to notify the MAPI PH of the process pushing MAPI URLs to be indexed
A new MAPI notification type has been introduced to facilitate shutdown scenarios for pusher stores. These stores will have to persist what has to be pushed since it may not be able to index everything before a shutdown occurs. When a store provider is a pusher it should send the following notification so that the MAPI PH knows which process it should watch for a given store. This way if the process is shut down or crashes the MAPI PH will immediately detect that and close (stop indexing) the store it opened.
#define fnevIndexing ((ULONG) 0x00010000)
/* Indexing notifications (used for FTE related communications) */
/* Shares EXTENDED_NOTIFICATION to pass structures below, */
/* but NOTIFICATION type will be fnevIndexing */
// Stores that are pusher enabled (PR_SUPPORT_MASK contains STORE_PUSHER_OK)
// are required to send notifications regarding the process that is pushing.
#define INDEXING_SEARCH_OWNER ((ULONG) 0x00000001)
typedef struct _INDEX_SEARCH_PUSHER_PROCESS
{
DWORD dwPID; /* PID for process pushing */
} INDEX_SEARCH_PUSHER_PROCESS;
BLOB structure associated with each URL when indexing
When pushing URLs to be indexed pusher stores should also create a blob with some information that will be used by the MAPI PH. This blob will be associated with each URL and will be sent when the URL is pushed to the indexer.
The format of the blob is the following:
DWORD dwVersion
DWORD dwFlags
ULONG cbProfileName
WCHAR wszProfileName
ULONG cbProviderItemID
WCHAR wszProviderItemID
Note that these values have to be written to the blob in the order shown above. The provider item ID should only be sent for folders to prevent opening extra folders to get this information.
Parameters
dwVersion
This is the version of the data being sent. Currently this value is 1.
dwFlags
Reserved for future use. Currently this should be 0.
cbProfileName
Size of the profile name in bytes. This information will be useful for the MAPI PH to know which profile to use when indexing the item.
wszProfileName
Null terminated Unicode string with the profile name.
cbProviderItemID
Size of the Provider Item ID in bytes
wszProviderItemID
Null terminated Unicode string with the provider item ID that uniquely identifies the item in the store.