Those 8197 Free/Busy Errors

A very good article on what causes the 8197 Free/Busy errors in the App event log posted by Jasper Kuria.

Do 8197 free/busy errors appear in your application log every 25 minutes? The actual text states:

Event Type: Error
Event Source: MSExchangeFBPublish
Event Category: General
Event ID: 8197
User: N/A
Computer: servername
Description: Error initializing session for virtual machine US-VS1. The error number is 0x80040111

One common reason for this error is the background task that polls the free/busy system folder to process updates cannot bind to a Global Catalog server (GC).  Unlike most Exchange tasks that use the DSACCESS mechanism to locate GCs, this free/busy task relies on the Windows operating system’s referral mechanism. DSACCESS and Windows referral use slightly different algorithms in determining an order of preference for GCs and so the free/busy task may not always be using the same GC that the majority of Exchange tasks are using. That is why this task may fail even when the Exchange server otherwise appears to be running normally. To troubleshoot these errors perform the following steps:

1. Determine what GC the free/busy thread is attempting to bind to. This can be found under the following registry key which is present only when the System Attendant service is running.

 HKEY_USERS\.DEFAULT\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles\ExchangeAdmin<server name> <some GUID>

Note that the above registry key is different from the

 HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\Exchange Provider\DS

key that is sometimes used to statically map DSACCESS to use one, and only one Directory Server (DS) or GC. I've seen this Exchange Provider key erroneously used to rule out ALL GC connectivity issues. It is only relevant for directory access involving the DSACCESS component. Once you establish what GC the free/busy task is attempting to bind to, use general network/connectivity troubleshooting to make sure that the GC is responsive and functioning normally. 

2.      If the GC is responsive and appears to be functioning normally, verify that there are no rogue entries for the GC in the Active Directory (a common cause of the 8197 errors). To do this execute this query from a command prompt:

ldifde -f output.ldf -d "dc=mydomain,dc=com" -t 3268 -p subtree -r  "(&(objectclass=*)(name=myserver))”

Of course substitute myserver for the name of your particular server and mydomain for your domain. Open the output.ldf file and look out for entries that look like this:

dn: CN=MYSERVER,CN=Computers,DC=mysubdomain,DC=mydomain,DC=com
…
distinguishedName: CN=MYSERVER,CN=Computers,DC=na,DC=mydomain,DC=com

In virtually all the incidents I’ve dealt with the rogue GC entry that the free/busy task was attempting to bind to was an entry in the Computers container like the one shown above. A valid GC should always have an entry in the Domain Controllers container and should look like this:

dn: CN=MYSERVER,OU=Domain Controllers,DC=mydomain,DC=com
…
distinguishedName: CN=MYSERVER,OU=Domain Controllers,DC=mydomain,DC=com

When both the good and the bad entry exist in the Active Directory, a DNS query for myserver may return the bad one and any attempt to bind to myserver will fail. To resolve the problem simply delete the rogue entry and flush DNS. Rogue entries of this kind are generally created when you stage a server in one domain, say, a sub-domain as a member server and then promote it to a DC/GC in a different domain. 

If the errors still appear in your application log after doing 1 and 2 above then you have a different problem. The other common reason for 8197 errors is installing Outlook on an Exchange server.  You should generally never do this (not even for ‘testing’ purposes) because of conflicts between the Exchange and Outlook versions of mapi32.dll, emsabp32.dll and emsmdb32.dll.

As a side note, Exchange’s Mailbox Manager also relies on the same Windows referral mechanism that the free/busy task uses to locate GCs.  Usually when the free/busy task fails with an 8197 error, Mailbox Manager will also fail and report the following errors.

Event Type: Error
Event Source: MSExchangeSA
Event ID: 9175
Description: The MAPI call 'OpenMsgStore' failed with the following error:
The information store could not be opened.
The logon to the Microsoft Exchange Server computer failed.
MAPI 1.0 ID no: 80040111-0286-00000000

Event Type: Error
Event Source: MSExchangeSA
Event Category: Mailbox Management
Event ID: 9200
Description: Failed to perform MAPI logon.

Not surprisingly, installing Outlook on Exchange servers has been known to break Mailbox Manager too.

By the way, the first time I worked on one of these issues it took a couple of nights and weekends of Starbucks powered live debugging to figure out the problem. When I finally explained the solution to the customer he asked me “Why do different Exchange tasks use different mechanisms to locate GCs. Isn’t this bad design?” Depending on how you look at it, yes and no. The Exchange team wanted to make it possible for administrators to install the Exchange System Manager (ESM) on non-Exchange computers without requiring them to load DSACCESS (a behemoth of a component) and so they implemented various tasks to use the underlying operating system’s mechanism to find GCs. Then why don’t all Exchange components simply use the Windows referral mechanism for consistency? In a word, optimization. When you are all things to all people (like the Windows referral mechanism is) you generally don’t do exceptionally well at any one given task. DSACCESS optimizes Exchange’s requests to the Active Directory and also acts as a cache for Exchange specific queries. This greatly improves Exchange’s performance but may also result in inexplicable 8197 errors when we assume that all tasks use DSACCESS—until now :)