Thursday, March 18, 2010

Free/Busy Federation Troubleshooting

I have had the pleasure of being the administrator of the very first organization to implement the new Exchange 2010 Free/Busy Federation (from now on I will call it F/B Fed) infrastructure last year during the Exchange 2010 TAP (Technical Adaption Program).  In doing so, I have been given the opportunity to work directly with a couple of the Microsoft Exchange Product Group members (thank you Ladislau and Matthias!!!) that guided me through the initial implementation and troubleshooting of Free/Busy Federation when it occasionally went awry.  I could probably write a small whitepaper on what I have learned, however for the purposes of this blog post, I wanted to delve into the latest issue I had. 

Recently, the public certificate we had been using for OWA, etc… and therefore for F/B Fed was going to expire and the cert vendor had made some changes to the UC certs they offered so we had to make a cert change, not just a renewal.  After we installed the new certificate and began using it for all the other web services (OWA, OA, EAS, etc…), we turned to F/B Fed and ran two commands with the intent of rolling to the new certificate.

Set-FederationTrust -Identity MyFederationTrust -Thumbprint <your new cert thumbprint here>

Set-FederationTrust "MyFederationTrust" –PublishFederationCertificate

The problem is, it didn’t work.  The new certificate didn’t get rolled to as it should have.  Instead, I received the error shown below.

An error occurred accessing Windows Live. Detailed information: "The request failed with HTTP status 403: Forbidden.".

+ CategoryInfo: ResourceUnavailable: (:) [Set-FederationTrust], LiveDomainServicesAccessException

+ FullyQualifiedErrorId: 7CDAC73F,Microsoft.Exchange.Management.SystemConfigurationTasks.SetLiveFederationTrust

Next, I validated that the new certificate was in fact valid and that the certificate was enabled for Server Authentication.

server-auth-sample So Far, everything looked ok, but we still couldn’t roll the cert properly and federation had stopped working as well.  ARGH..

After a bit more trial and error, it had seemed like the Set-FederationTrust command shown earlier had finally worked, at least it didn’t give me an error when I ran it, however, F/B Fed still wasn’t working and when I ran Test-FederationTrust –Verbose, I received the following error in response.

RunspaceId : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Id         : OrganizationPreviousCertificate
Type       : Error
Message    : Certificate referenced by property OrgPrevPrivCertificate in the FederationTrust object is expired.

With the help of Matthias, I ran the following script in order to attempt to push the old certificate completely out of the Federation system.

$a = Get-FederationTrust

Set-FederationTrust -Identity $a.Identity -Thumbprint $a.OrgPrivCertificate

Set-FederationTrust -Identity $a.Identity –PublishFederationCertificate

Unfortunately, when I ran the second command, I received a new error.

Federation certificate with thumbprint "C54359E291F10213…" must have a unique Subject Key Identifier.  The Subject Key Identifier "1A29F0C8C62971EA524BE4…" is already used by the certificate with thumbprint "C54359E291F10213…".

+ CategoryInfo: InvalidArgument: (:) [Set-FederationTrust], ProvisionerConfigException

+ FullyQualifiedErrorId: 4CFC5CA6,Microsoft.Exchange.Management.SystemConfigurationTasks.SetLiveFederationTrust

So, it seemed at the time that the issue was more of a security one due to the beta we are running for Service Pack 1, so we tried a different approach.

$a = get-federationtrust

$b = "LDAP://" + $a.DistinguishedName

$c = [ADSI]$b

If ($c.msExchFedOrgPrevPrivCertificate -ne $null) { $c.PutEx(1, "msExchFedOrgPrevPrivCertificate", 0) }

If ($c.msExchFedOrgPrevCertificate -ne $null) { $c.PutEx(1, "msExchFedOrgPrevCertificate", 0) }

$c.SetInfo()

I ran that script (without error) and waited for AD to replicate.  Afterwards, I ran Test-FederationTrust –Verbose again, this time with a slightly different error, yet still related to the “msExchFedOrgPrevPrivCertificate” attribute.

RunspaceId : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Id         : OrganizationPreviousCertificate

Type       : Error

Message    : Unable to find certificate referenced by property OrgPrevPrivCertificate in the FederationTrust object.

Hmmm…  that is interesting, now I don’t have a value in that attribute at all!  So I checked that by running Get-FederationTrust | fl and sure enough, this attribute was empty from Exchange’s point of view.  However, not fully convinced, Ladislau recommended I run the script below just to ensure it really was missing from AD.

$a = get-federationtrust

$b = "LDAP://" + $a.DistinguishedName

$c = [ADSI]$b

$c | fl * -force

And of course, it was actually missing from AD as well….

Come to find out, I had hit a new unknown bug on cert rolling and had to run this final script to set the msExchFedOrgPrevPrivCertificate attribute and get F/B Fed working again.

$a = get-federationtrust

$b = "LDAP://" + $a.DistinguishedName

$c = [ADSI]$b

$c.msExchFedOrgPrevPrivCertificate = $c.msExchFedOrgPrivCertificate

$c.SetInfo()

Now, when I run Test-FederationTrust –Verbose I get a “Success” on all tests!  And our users are happy because Free/Busy Federation is once again working as advertised.  I hope these little insights are helpful to others when they use their favorite search engine to find answers to their own Federation issues. 

Until next time…