How to troubleshoot the OAB Generation process

  • Article

I thought I would take a few minutes to post some information on how to troubleshoot OAB Generation problem. There are two really basic things that must be understood before troubleshooting any process:

 

  1. Understanding how the process works before you troubleshoot it. If you do not know how the process works, how are you really sure that it’s broken and what you see is not expected behavior?
  2. If you suspect problems, what evidence do you have to indicate that there is really a problem?

There are way too many situations when things are broken and people just don’t know where to start. One good place to start is with the Application and System logs. The first thing to do when troubleshooting an OAB generation problem is to stop and ask yourself is this and OAB Generation problem and if so what kind?

One way to verify this is to turn up diagnostic logging to maximum or expert, and then rebuild your Offline Address List. This will be done two different ways depending on the version of Exchange you are running (Exchange 2003 or Exchange 2007).

 

1. How to turn up diagnostic logging in Exchange 2003

 

A. Open the Servers container.

B. Right click and select properties.

C. Select the Diagnostics Logging Tab.

D. Under Services select 'MSExchangeSA'.

E. Under Categories select 'OAL Generator'.

F. Select Maximum and click ok.

 

How to turn up diagnostic logging in Exchange 2007

A. Please see this blog: https://blogs.msdn.com/dgoldman/archive/2006/08/26/725860.aspx

2. Now after you have done this you will want to save out the application log and then clear it out. You don’t need to have all the other information in there clouding the real problem.

3. Rebuild your Offline Address List. Again this will be different based on the version of Exchange you are running.

4. How To Rebuild an Offline Address List in Exchange 2003

A. Open the Recipients container.

B. Right click and select properties for your Offline Address List.

C. Select 'Rebuild'.

<>

How To Rebuild an Offline Address List in Exchange 2007

A. You can do this via the Microsoft Exchange Mangement Shell (Powershell) - Update-OfflineAddressBook –Identity “Your Offline Address List”
B. You can do this via the Exchange 2007 ESM.

Now that you have turned up diagnostic logging and rebuilt your Offline Address List, look in the application log and see how far you got!! If you do not see any errors there is a good chance that it completed successfully and you are looking at an Outlook client side download issue.

Here is a list of events from start to finish for a successful Address List generation in Exchange 2000. Exchange 2003 has a few more descriptive events, but you can expect to see the same thing.

9106 - Offline Address List generation has been started.
9117 - OALGen successfully opened a connection to the DS which will supply the current Address Lists.
9121 - OALGen successfully loaded the hierarchy table for /. (\Global Address List)
9131 - OALGen is attempting to open the Public Folder which stores Offline Address Lists.
9132 - OALGen has successfully opened the Public Folder which stores the Offline Address Lists.
9133 - OALGen is attempting to create or open the sub folder 'OAB Version 2' in the OAL Public Folder.
9134 - OALGen successfully created or opened the sub folder 'OAB Version 2' in the OAL Public Folder. 9122 - OALGen successfully removed any OAL-related public folders which are no longer needed. This normally occurs because an OAL is no longer being generated. No actual cleanup may have occurred, if all former OALs are still being generated.
9108 - Offline Address List is being generated for \Global Address List.
9123 - OALGen successfully generated templates files. (\Global Address List)
9118 - OALGen successfully read all 7502 objects in the DS and wrote them to files on disk.
9119 - OALGen successfully computed and wrote to disk the index for file ANRdex.oab. (\Global Address List)
9119 - OALGen successfully computed and wrote to disk the index for file RDNdex.oab. (\Global Address List)
9120 - OALGen successfully wrote file browse.oab to disk. (\Global Address List)
9131 - OALGen is attempting to open the Public Folder which stores Offline Address Lists.
9132 - OALGen has successfully opened the Public Folder which stores the Offline Address Lists.
9134 - OALGen successfully created or opened the sub folder 'OAB Version 2' in the OAL Public Folder. 9135 - OALGen successfully opened the public folder containing the previous version of this OAL. (\Global Address List)
9136 - OALGen successfully opened the message containing the previous version of this OAL. (\Global Address List)
9137 - OALGen successfully set the property view of the previous version of this OAL. (\Global Address List)
9138 - OALGen successfully found the message containing the previous version of this OAL. (\Global Address List)
9111 - OALGen is calling into emsabp32.dll to download the previous version of this OAL. (\Global Address List)
9112 - OALGen has downloaded the previous version of this OAL successfully. Differences between the previous and the current version will be computed. (\Global Address List)
9139 - OALGen will write the previous version of this OAL to temporary files in C:\WINNT\TEMP\.
9113 - OALGen is comparing the previous and new version of the OAL to determine what changed since OALGen was last run. (\Global Address List)
9114 - OALGen successfully compared the previous and new version of the OAL. OALGen will now generate the changes.oab file that clients will download for a differential download. (\Global Address List)
9115 - OALGen successfully generated the changes.oab file that clients will download for a differential download. (\Global Address List)
9131 - OALGen is attempting to open the Public Folder which stores Offline Address Lists.
9132 - OALGen has successfully opened the Public Folder which stores the Offline Address Lists.
9133 - OALGen is attempting to create or open the sub folder 'OAB Version 2' in the OAL Public Folder.
9134 - OALGen successfully created or opened the sub folder 'OAB Version 2' in the OAL Public Folder. 9125 - OALGen successfully generated and posted to a public folder the OAL for clients newer than Exchange 5.0. (\Global Address List)
9107 - Offline Address List generation has finished.

From the application log events you can make out the OAB generation process has completed. Here is a down and dirty recap and are the core spots to look for when troubleshooting.

1. The System Attendant will make a DSAccess call to the Active Directory to obtain information about the Offline Address List we are going to rebuild.

2. OABGen.dll will make a connection to the active directory using the System Attendant's profile to read from the Address List containers. For more information on how we create this profile and gather the connection information, see this blog: How does OABGen know what domain controller to connect to?

NOTE: If you are not sure what domain controller you are connecting to, you can take a Netmon trace and look for NSPI traffic. In Exchange 2003 we log the server we are connecting too. You have to remember that you will not see any traffic you are on an Exchange server that is also a domain controller. All traffic will be in the form of a NSPI:QueryRows call from the Exchange server to the domain controller.

3. Read the hierarchy table from the public folder information store on the server we are generating the OAB on. All of the objects will be in this AB Hierarchy.

4. Open the OAB v2, v3 or v4 folders.

5. Check for messages and folders that do not match or are no longer needed.

NOTE: Steps 3, 4 and 5 we are connecting to the public folder information store. If you are failing with in here you need to make sure you are actually getting to the information store and or folders. You can use a few tools to test this out:

NOTE: If you are going to run MFCMapi or OABInteg on an Exchange 2007 server you will need to install the Standalone MAPI/CDO components. Exchange 2007 no longer ships with the client side binaries (EMSMDB32.dll, EMSABP32.dll, etc). For more information see this blog: Standalone installation package for MAPI and CDO

6. Select the address list to generate.

7. Read in all of our entries from the active directory.

8. Write our information to disk (the system temp path).

NOTE: Filemon.exe is a good tool to use when monitoring the file creation. It can be downloaded from here: https://www.sysinternals.com/Utilities/Filemon.html

9. Open the information store and download the previous messages and attachments.

9. Compare our data so we can generate our changes.oab files.

10. Generate the changes. This will create change.oab files for Version 2 and 3a, binpatch.oab files for Version 4.

11. Open the folders in the information store and post the new data.

12. Repeat this process for OAB v2, v3 and v4. (v4 will only be available if you have Exchange 2003 Service Pack 2 installed).

13. We are finished with the rebuild process.

So from looking at this you can see that there are four main places that we can have problems.

1. Connecting to the active directory:

  • Having a stable infrastructure is a MUST in order for OAB generation to work successfully.
  • If you active directory domain controllers are not replicating as expected you might see some problems here.
  • Replication latency and sort orders can also cause problems. These will be seen when downloading the OAB files to the client and looking at the address list from within outlook.

2. Connectivity to the public folder information store:

  • Permissions and database problems can affect this.
  • Permissions on the /Global Address List and Address Lists in the Active Directory
  • Networking problems.

3. Disk access:

  • Virus scanning the system directories can cause problems when trying to write a few hundred files to disk.
  • Not enough disk storage on your c:\. We use the system temp path as a temp location when we are building our OAB files.

4. Check your online maintenance schedule for your information stores:

  • OABGen.dll is used to rebuild your address lists and the recipient update service rebuilds the memberships and stamps the objects with policies that are needed to to add the objects to the Global Address List.
  • When you create an offline address list, it will reside in two places [The active directory and the information store]. The information store will query the active directory via DSAccess to look for all new Offline Address Lists that were created. Once new address lists have been found the information store will create the root folder. The OAB generation process will create the sub folders when we need to post to them.
  • You public folder will not get created until Online maintenance has completed, and or dismounting all store and restarting.

I hope this sheds some light on how to troubleshoot the OAB Generation process. I have also included some useful Knowledge Base articles have great information.

Dave