OAB sort orders and character handling

Borrowed from Neil Shipps Blog...

In Exchange 2000 and earlier, 10% of Exchange users used the Offline Address Book was the statistic that was most often repeated to me. But with Exchange 2003 and Office 2003, that number has gone way up due to Outlook's usage of Cached Mode Exchange. This means that everyone that uses Outlook in cached mode now downloads the Offline Address Book or OAB, and they may get a slightly different experience than they were used to.

Before Cached Mode Exchange, clients would get their address books directly from the Active Directory. As long as the global catalogs they were using had the client language installed (see KB http://support.microsoft.com/default.aspx?scid=kb;en-us;301314), the Active Directory would sort the GAL for browsing using the correct sorting locale and return it to the client rendered in the right code page. When the client actively searched for entries, again, the AD would use the sort locale of the client to compute the result. When the OAB server generates the OAB files, it reads the entries from the AD sorted in the default sort order of the OAB server and builds the OAB search files using that same sort order. Now in cached mode, since the OAB is sorted according to OAB server and not the client, it may not match the clients language locale. This means that users may not be able to find entries in the GAL as easily because they're not sorted in the way the user expects.

By default, Exchange Server creates a single default OAB with the Global Address List on the first server installed in the organization. This works well if your users all have the same language set as default on their client computers. But for those organizations who must support different client languages it now gets complex. The OAB generation process in Exchange Server 2003 can build two different versions of the OAB: a Unicode version and an ANSI version. The Unicode version reads the Active Directory recipients in Unicode and keeps the data as Unicode when stored on the client. It's up to the client to render the characters in the right code page. However for the ANSI files, the OAB server renders the Unicode AD characters using the default code page of the server. So if your OAB server is running on a US English Windows server and you have some Active Directory recipients with non-western characters in their properties, you're going to get question marks for those characters in the OAB data for the ANSI version of the OAB.

For the display names that can't be rendered in ANSI there is a work around. If the AD or the OAB finds that characters in the displayname can't be rendered correctly in the current code page, they will then substitute the "simple display name" attribute for the display name. If you have filled out this attribute then that text will show up instead of the original display name in the address book and on the property pages. If it hasn't been set, the alias name will be used. However, the records will not be resorted. The simple display name will show up where the display name would have sorted to. Since the display display name or alias may not have any relation to the display name, then the address book may not appear to be in the correct order for those entries.

This leads us into sort order problems. Alphabetical English is not the only way that words are sorted. Many languages use different sorting rules. The far-east languages are the most removed from English, but even western European languages use slightly different sorting rules. Letters with diacritical marks may sort differently: 'A' with an umlaut may come after 'Z', double 'ss' may sort the same as a single 's', or 'ch' is considered a different letter that comes in-between 'c' and 'd'.

When you have users who are geographically diverse, they may require that the OAB files they use be sorted differently. Address Name Resolution may not be able to find entries and type-down in the Address Book pane may not work. This is because the user's client is using one set of rules to search a list of words sorted using an entirely different set of rules. The user could enter 'A' with an umlaut and they may be taken to the end of the address book when the entries they wanted are at the beginning. Or when entering a display name in Outlook and trying to resolve the name comes up with no matches found because the OAB was sorted with a different sort order. To improve this situation, in Exchange Server 2003 the OAB server stamps the OAB with the sort order that was used to create the files so the client knows what search rules to use when searching the OAB. However this only works for Outlook 2003. Older clients may still use different search rules.

To completely solve this issue you have one option: install a different Exchange OAB server with its own OAB for each client locale you want to support. Each server would have the system locale set to the locale of the client language you want to support. Once you've done this you now need to change the OAB settings for each user who requires a different sort locale. If you can move all your users who require a specific language to one Mailbox Store, you can change the default OAB setting for the Mailbox Store to the OAB that is sorted in the correct locale. If you can't put all the users who require the same sort order on the same Mailbox Store, you can set the user's OAB on the user object itself. See KB http://support.microsoft.com/default.aspx?scid=kb;en-us;275203

Since the OAB is now used by a larger part of the customer base, Microsoft is putting more resources on documenting the OAB. Microsoft has a good whitepaper on Offline Address Book best practices which covers OAB issues including sort orders. It can be found here: http://www.microsoft.com/technet/prodtechnol/Exchange/guides/OffLineAddressGuide

ExBPA v2.0 Is Here

ExBPA v2.0 has been released and you can grab it here. This version has support for MOM 2005 which I wrote about yesterday.
There are many improvements, such as the support for MOM 2005, multiple language suppport, DNS collector, scheduling capabilities, a new 'best practice' category, an improved scanning scope, etc, etc.
You can find out about all the new features of this version via the readme.

Exchange Web Releases due this week

From my buddy Joe!...
Just thought you may like to know what is coming this week:

Exchange Web Release 1.2 Scheduled to RTW March 15, 2005!
WR 1.2 Updates
Web Release 1.2 is scheduled to RTW March 15th, 2005. This message contains information on what is new and updated.
Overview of changes in WR 1.2:
WR 1.2 Updates. 1
Overview of changes in WR 1.2: 1
Summary: 1
New Tools: 1
ExMon. 1
New versions: 2
ExBPA v2. 2
Doc and tools changes: 2
LoadSim.. 2
Doc changes only: 2
UTD Notifications Binding Cleanup. 3
JetstressUI 3
MSSearch. 3
Exchange Profile Update. 3
Exchange Domain Rename Fix-Up. 3
ExMerge. 4


Summary:
New Tools:
ExMon
Description
Exchange User Monitor (ExMon) for Microsoft® Exchange Server enables administrators to view an individual user's usage and user experience with Exchange Server 2003. With this tool, you can gather real-time data for operating servers, to attain a better understanding of usage patterns of clients, and for future planning.
Additional Information
Please see the following documents for more information about ExMon:
Using_ExMon2.doc
Using_ExMonInternal.doc


New versions:
ExBPA v2
Description
The newest version of ExBPA includes numerous database updates as well as some very impressive enhancements including:
MOM 2005 support
Localized into all server languages
Scheduling capabilities
Additional Information
There are many more enhancements to this very useful tool. For more information, please see the previous messages sent by Paul Bowden and Shawn McGrath.

Doc and tools changes:
LoadSim
Description
Documentation changes:
Expected database size after default installation for 500 users.
Screenshot updated to show LoadSim 2003 values instead of LoadSim 2000 values.
Elaboration on what the generated score means.
Elaboration for remote configuration.
Running LoadSim 2003 with any version of outlook other than Outlook 2003 or Outlook 2000 is not supported.
Removed lsperf.reg import section from the LoadSim docs. The registration of performance counters is handled within the MSI setup, and we don't drop the lsperf.reg file anymore.

Application Change:
Invalid Delivery Time Warming changed to “debug” level because of erroneous messages as a result of initialization.
Bug Numbers
253112 DCR: Expected Database size needs to be documented or have LoadSim estimate it.
253113 Loadsim2003 doc still shows Loadsim2000 values and needs to get updated
253139 document why loadsim2003 does not work with any other versions of Outlook besides Outlook 2003
251951 LoadSim 2003 new warning to be documented
253188 Need to remove lsperf.reg import section from the LoadSim docs



Doc changes only:
UTD Notifications Binding Cleanup
Description
Updated documentation to remove references to “Select All” as a result of a UI change.
Bug Number
230515 Up-to-Date Notifications Binding Cleanup Tool: Documentation Update Required


JetstressUI
Description
Documentation modified to give guidance on configuring JetstressUI.
Bug Number
252357 JestressUI: documentation is not clear how the input parameters relate to the end database sizes
252792 JetstressUI: documentation needs to be updated to call out reasoning for 6 databases and how it doesn't have to match production


MSSearch
Description
Documentation modified to describe using “cscript.exe /h:cscript” or “wscript.exe /h:cscript” so that the usage is displayed on the command prompt instead of in dialog boxes.
Bug Number
252870 MSSearch_Admin tool - suggestions


Exchange Profile Update
Description
Documentation modified to state that you can not run Exchange Profile Update on a Terminal Server. There were also some link and heading fixes.
Bug Numbers
253024 ExProfRE is not supported on Terminal Server.
254340 Exchange Profile Update Tool doc: fix broken link reference

Exchange Domain Rename Fix-Up
Description
Documentation changes to reflect that
The Exchange Domain Rename Fix-Up Tool was first supported in Exchange 2003 SP1, NOT the RTM version.
The Exchange Domain Rename Fix-Up Tool will verify servers whose DNS name is part of the domain. If you have static names that do not contain the domain name, then they will NOT be reported with /verify. Customers should manually ensure that they do not have these or that they handle these manually.
Applying the restore.ldf file will undo the changes to the AD caused by applying the changes.ldf file.
Bug Numbers
235167 Domain Rename: "Exchange Domain Rename Fix-up.doc" should also state that E2K3 (Ti RTM) is supported with Domain Rename.
241567 Domain rename: after the second rename XDR-fixup /Verify does not report verification status of exchange server with statically assigned DNS name.
250213 XDR-Fixup doc needs to mention use of restore.ldf for reverting changes.ldf,

ExMerge
Description
Numerous small documentation bugs fixed
Bug Number
254939 ExMerge Tools Doc: Updates for WR1.2. This is a tracking bug for the numerous changes.

ExBPA Management Pack for MOM 2005

Microsoft has released the Exchange Best Practices Analyzer Management Pack for MOM 2005.
Overview:
The Microsoft Exchange Server Best Practices Analyzer (ExBPA) Management Pack can be used to deploy the ExBPA tool on computers running Microsoft Exchange Server 5.5, Exchange 2000 Server and Exchange Server 2003. Using a timed event, the tool will run automatically on each server and write any identified performance, scalability and availability issues to the Windows NT Event Log.

The ExBPA Management Pack will interpret these events and generate the appropriate alerts on the MOM Management Console.The ExBPA Management Pack works in tandem with the ExBPA tool:

• Identification of Exchange server configuration issues which could result in poor performance, scalability and unplanned downtime
• Automated analysis and root cause identification
• Support for third-party software and hardware
• Self-updating database and help content
• Links to over 500 articles on the Web
• Works with Exchange Server 2003, Exchange 2000 Server and Exchange Server 5.5 (in mixed mode topologies

Microsoft Renames 'Istanbul'

Microsoft has rechristened its "Istanbul' corporate instant-messaging client "Office Communicator 2005." The new client requires Service Pack 1 for Live Communications Server 2005 in order to provide the enterprise messaging, VOIP and web-conferencing functionality that Microsoft rolled out last fall. Microsoft also unveiled on Tuesday an upgrade to its Live Meeting web-conferencing service, dubbed Live Meeting 2005.

Exchange Tidbits

Evan Dodds posted recently a few answers to common questions.  Here are a few of them:

Couple of Exchange tidbits I’ve been saving up for a while. None of them are on my usual topics (Exchange clustering, site consolidations, etc) and they are not large enough individually to warrant their own post, but hopefully they’ll help you out.

1) You can force the Exchange 2003 SP1 RUS to stamp secondary email proxy addresses onto an existing user account without having to do “apply now” on the recipient policy. This is principally for cross-forest migrations, but may also work for users who haven’t been moved cross forest. KB.820381 covers the cross-forest scenario, but you can also set the GUID manually on the user’s msExchPoliciesIncluded attribute: {23668AD4-4FA1-4EE8-B2BB-F94640E8FBA0}.

2) NTDSNoMatch doesn’t have to be set on Custom-Attribute-10. Everyone refers to extensionAttribute10 when discussing NTDSNoMatch (see KB.274173, for instance). Little-known fact is that it doesn’t have to be attribute #10. If you’re using attribute #10 for something else, just put the “NTDSNoMatch” value into any of the 15 extensionAttributes and the ADC will pick this up transparently. Thanks to Alex Seigler for this find.

3) LegacyExchangeDN may add a random 8 digit number to ensure uniqueness. You may notice that sometimes the LegacyExchangeDN value associated with a user is of the format /o=org/ou=site/cn=recipients/cn=username######## rather than the expected /o=org/ou=site/cn=recipients/cn=username. This happens because the LegacyExchangeDN serves as the unique X500 identifier in the Exchange 5.5 directory, and as such, has to be unique. If there is any collision of LegacyExchangeDN values when the user is being mailbox enabled, eight random numeric digits are appended to the proposed LegacyExchangeDN and it’s checked again for uniqueness. Note that it really doesn’t matter what your LegacyExchangeDN is, so long as it’s properly associated with the user and is unique (ie - don’t bother trying to go back and “change” these unique-ified values)

Update for Outlook 2003 Junk Email Filter (KB892236)

Microsoft has posted another Junk E-mail Filter update for Microsoft Office Outlook 2003 with a more current definition of which e-mail messages should be considered junk e-mail. This update was released in March 2005.

You can get specific information about this update in the Microsoft Knowledge Base article Description of the Update for Outlook 2003 Junk Email Filter (KB892236).

Sender ID ready

SenderID is now ready for deployment.  Now lets convince the rest of the world to start using it.


see:

Implementation Tips for the Sender ID Framework—Creating Your SPF Record

Sender ID: Sender ID Overview

Sender ID Framework SPF Record Wizard

Microsoft Unveils Next-Generation Real-Time Collaboration Client

Real-Time Collaboration Client Will Strengthen Connection Between Customers And Their Telephony, Line-of-Business and Productivity Applications; Beta Program Open for Nominations

Today in his keynote address at the Fall 2004 VON Conference & Expo, Anoop Gupta, corporate vice president of the Real-Time Collaboration (RTC) Business Group at Microsoft Corp., outlined Microsoft's real-time collaboration vision and strategy for integrated communications, and announced plans for a next-generation client. Building on the recent success and rapid adoption of Microsoft® Office Live Communications Server and Microsoft Office Live Meeting, Gupta introduced the newest member of the Microsoft Office System, code-named "Istanbul," that will provide integrated communications capabilities including instant messaging, extensible presence, PC-based voice and video, and telephony integration.

"Customers depend on real-time communications to make more informed, timely decisions, and to stay better connected with their customers and grow their business," Gupta said. "With 'Istanbul,' Live Communications Server and Live Meeting, we are providing pervasive real-time collaboration capabilities and empowering our customers to work together more effectively. 'Istanbul' represents a milestone by integrating various modes of communications in one unified desktop experience."

......
The "Istanbul" beta program is open for nominations. Those interested in participating should contact their Microsoft account manager or sales representative for instructions on how to nominate their organization. "Istanbul" is scheduled for release in the first half of 2005. Full Story At Source

Exchange Server 2003 Security Hardening Guide (updated)

Updated February 2005. The download package for this guide includes important security templates. These templates were updated November 2004. This guide walks you through the process of hardening your Exchange 2003 environment, including configuration recommendations and strategies for combating external threats.

Read the entire Exchange Server 2003 Security Hardening Guide online or download the Exchange Server 2003 Security Hardening Guide . This guide is designed to provide you with essential information about how to harden your Exchange Server 2003 environment. In addition to practical, hands-on configuration recommendations, this guide includes strategies for combating spam, viruses, and other external threats to your Exchange 2003 messaging system. Important: Since the previous version of this guide was released, the following new topics have been added and are available only online:
• Running Exchange Server 2003 Clusters in a Security-Hardened Environment
• How to Run Exchange Server 2003 Clusters in a Security-Hardened Environment Download At Source

Live Meeting 2005 for Intranet deployment

Live Meeting 2005 Add-in: Microsoft Office Live Meeting Intranet Portal

The Microsoft Office Live Meeting Intranet Portal makes it easier to deploy and manage Live Meeting.

 

Live Meeting 2005 Intranet Portal Administrator’s and Programmer’s Guide

The Live Meeting 2005 Intranet Portal Administrator’s and Programmer’s Guide provides information to help administrators and developers to deploy, use, and customize the Live Meeting Intranet Portal.

Live Meeting 2005 Document: Live Meeting 2005 Deployment Guide

This guide provides the information you need to plan, understand, and perform a deployment of the Microsoft Office Live Meeting 2005 console in your environment.

And here are some Exchange 12 answers!

In a post from KC Lemson:

A few weeks ago, we asked you for your list of questions about E12 and said we'd pick a few and post answers on the one year anniversary of this blog, February 9th. Well as you can see, I totally missed that date :-) Very sorry about that and I'd be happy to list off my excuses, but let's get on to the good news instead: for the foreseeable future, we are going to have a monthly post about E12, the content will usually be based on the questions we get from you. So without further ado, here are answers to the first set of questions. Please add any additional questions as comments on the E12-focused posts and they'll be added to the list for consideration for answering the next time around. We won't be able to answer everything, depending on the state of the feature in question, as we only want to talk to you about features that are very baked. Without further ado:
 
Question #1: Lots of people asked about the 32k rules limitation:  "As someone else mentioned how about improvements to server side rules (32k limit)", "Since I ask this every time I talk to Microsoft developers :), any news on when will the 32k server side rule limitation be resolved".
 
Answer #1: I'm happy that I get to be the one to tell you that the 32k rules limit is gone... It's now 64k! Just kidding!! We are still determining what the default will be (there's a tradeoff in performance of course, the more processing you ask the server to do on inbound mail, so we want to choose a smart default), but the administrator will be able to change the limit.
 
Question #2: I'd like to see a unified messaging function in E12 (facsimile, voice,...).
 
Answer #2: UM is absolutely an area we are investing in, as you can see from some of the articles about it, such as this one from eWeek. If you call me at Microsoft today and leave me a voicemail, it gets routed to my inbox using only Microsoft technology.
 
That's it for right now, and look for a post next month with some more questions and answers about E12.

Exchange 12 To Ride a Faster JET

From an article in RedmondMag.com:

Exchange 12 To Ride a Faster JET
Microsoft is finally providing specifics on the next version of Exchange.

March 2005 • by Scott Bekker

Microsoft's long silence on Exchange is over.

In January, Microsoft publicly laid out plans for the next release of Exchange. Microsoft is calling the next version "Exchange 12" or "E12." The code-names emphasize the close ties to the next version of Office, code-named "Office 12."

Usually Microsoft starts talking about the next version of a product as soon as, or even before, the latest version ships. But since Exchange Server 2003 went gold in June 2003, Microsoft hasn’t said much on the record about its successor.

The big problem is that Microsoft had to figure out what to do next with Exchange. Exchange 2003 was an incremental release, essentially upgrading Exchange 2000 Server to take advantage of some useful plumbing changes in Windows Server 2003. They included security improvements, eight-node failover clustering and Volume Shadow Copy services.

For years the company has been toying with the idea of taking the data store from SQL Server and porting it to Exchange, replacing the JET storage engine that the messaging server currently uses. The effort was code-named "Kodiak," a vaguely defined future version of Exchange that would use the new data store. The design goal reflected a wider, long-standing effort within Microsoft to standardize the data store across major products. Microsoft had ambitious plans to use the SQL Server storage engine in the next version of Windows, code-named “Longhorn,” and Exchange. Cracks in that plan emerged in June, when Microsoft said Kodiak was off the table. A few months later, unified storage, known as WinFS, was pulled from Longhorn, too.

The JET storage engine will power Exchange 12 just as it did Exchange 2000 and Exchange Server 2003. A Microsoft spokesperson downplayed the technological role in JET's survival. "The decision to ship the next version of Exchange with JET was based on many factors, but the primary reason was customers. Staying with JET will mean customers will not be faced with the migration work associated with moving to a new store."

Customers have been unhappy with JET in scalability, high availability and developer hooks. JET is also old: Related to the JET engine used in the Microsoft Access database, it's a generation behind the SQL Server engine. To address these issues, JET is getting a serious overhaul for Exchange 12.

First, JET is getting x64 support (AMD64 and Intel EM64T), to allow greater scalability. No word as yet on Itanium support, but unlike SQL Server, Exchange would probably get as much scalability benefit as it could use from the 64-bit extensions. Microsoft is leaning more and more heavily toward supporting x64 in many products, and company officials predict most new server chips will ship with 64-bit extensions within the next year.

Exchange Web Releases

Microsoft won't have a new version of Exhange this year, but the company plans to push out several new technologies in a service pack and in Web releases: A storage sizing assistant An SMTP configuration diagnosis tool Public folder usage and analysis tools Integration of the Exchange Best Practices Analyzer with MOM 2005 Integration of the Sender ID Framework with SP2 — Scott Bekker

To address high-availability requirements, JET will support log shipping, a feature Microsoft built into SQL Server 2000. The technology allows Exchange to send its log of changes to another server, off-site if necessary, at set intervals. The process is the poor man’s server’s clustering—more likely to miss the most recent transactions but less expensive. The nod to developers comes with adding Web services to reduce the complexity associated with writing to multiple application programming interfaces.

Above the data level, broad themes of the next release are improved end-user productivity, total cost of ownership, manageability and secure messaging. The planned end-user productivity boost comes in part from an investment in unified messaging. Exchange 12 is slated to have an architecture within Exchange for e-mail, voice mail and fax.

Any new version of Exchange would come after Microsoft ships the "Istanbul" client for Live Communications Server 2005. That client integrates telephone with a Windows Messenger interface and the technology could provide a bridge to unified messaging in Exchange (see December 2004’s Redmond Roadmap, “The Road to Istanbul”).

Other ways Microsoft plans to improve user productivity for Exchange 12 include investments in the mobile device experience, Outlook Web Access and efficient and reliable meeting scheduling. Microsoft is working to reduce total cost of ownership and manageability for Exchange Server with scripting, the Web services APIs and improved search functionality.

Exchange Joins Microsoft’s Billion-Dollar Club

While Microsoft has begun talking about new versions of Exchange, the company is raking in cash from the old versions. Microsoft's Exchange Server business crossed the $1 billion revenue threshold in fiscal 2004 (July 1, 2003-June 30, 2004), the company disclosed recently. That fiscal year coincided almost perfectly with the launch and first 12 months of availability for Exchange Server 2003. Other Microsoft products that are good for a billion dollars or more in revenues each year are the Windows client operating system, Microsoft Office, the Windows server operating system and SQL Server. Microsoft’s total revenues for fiscal 2004 reached $36.8 billion. — Scott Bekker

Security is a huge messaging issue that Microsoft could address in a number of ways. For now, Microsoft officials say they are channeling resources into message "hygiene" at the perimeter and creating a privacy compliance infrastructure. The message hygiene technology is a remnant of the Exchange Edge Services 2005 project, which was broken up into components that will ship with Exchange Server 2003 Service Pack 2 and Exchange 12.

Dave Thompson, corporate vice president in charge of Microsoft's Exchange Server Product Group, says the release of Exchange 12 will be loosely tied to Office 12, but won't ship at the same time. Office 12 is unofficially scheduled for 2006. Exchange 12 may come in 2006 or 2007.

For now, the Exchange team is faced with a situation similar to the SQL Server team—a long time between releases. While the Exchange team may not be borrowing the SQL Server engine, it is borrowing a page from the SQL team's playbook in dribbling out new features and tools in service packs and Web releases (see "Exchange Web Releases," above ).

Exchange 12 Presentation

Adam Field has posted details of a LiveMeeting recording that you can listen to, in which Dave Thompson, Exchange Server Corporate VP, talks about the future roadmap of Exchange, and what can be expected with Exchange 12... it's well worth a listen.

The instructions for listening are:

To view the LiveMeeting recording, go to http://www203.placeware.com/cc/mvp50/viewRecordings, Enter Your Name and enter UKESP for organization. Use Meeting ID: HM54RH, Meeting Password: FZC29R.

How does move mailbox really work?

Evan Dodds has posted this high level desription of how mailboxes are actually moved.

Move mailbox is the best, supported way to move mailbox data between Exchange servers and update the directory object. It’s been around for ages and has been improved with each version. In Exchange 2003, for instance, the mailbox moves can now be scheduled and are multi-threaded to dramatically improve performance. Exchange 2003 SP1 added the ability to move mailboxes cross-site while still in mixed mode.

There are a number of resources on how to do move mailbox between Exchange servers (KB.224975 and KB.328810 are two good examples), but what’s missing is a good high-level description of what goes on behind the scenes to make it all happen. This post focuses on Exchange 2003, but much of this applies to earlier versions as well. There’s a bunch of additional steps required for cross-site moves, but those are covered in other places.

After you click the last “Next” in the move mailbox wizard what happens?

All of the data provided to the wizard is processed and the move actually begins. There are a number of distinct steps we’ll talk about that are part of the mailbox move.

1) Wait for the scheduled time to arrive

Exchange 2003 provides a new function that allows you to schedule the time/date of the mailbox move. If you use this new feature, a second GUI window is opened up automatically to keep running until the scheduled time arrives and then perform the moves. This means that you don’t have to leave the original GUI window open after the scheduled move is set.

In any event, eventually the time of the move will be reached – either right away in the same GUI window or at some later point in the secondary GUI window – and the move will begin.

2) MAPI connection is opened to the source and destination servers

It’s important to understand that MAPI is used to facilitate the actual move. All Exchange servers have a basic server version of MAPI installed which is sufficient for the sort of MAPI operations that are required of the server, but note that this is a very different version of MAPI than is included with Outlook (it is because of conflicts between these two MAPI implementations that installing Outlook on an Exchange server is not a supported scenario).

The dependence on MAPI to move the data means that the requirements for MAPI connections must also be met for mailbox moves. For instance, an RPC connection must be made from the workstation or server running the move GUI to both the source server and to the server which is the destination of the move. If network ports are blocked between these systems, the MAPI connection won’t work. Similarly, if there is high latency or poor bandwidth availability in the network connections between these systems, this will also lead to poor mailbox move performance or failed moves.

3) Set PR_IN_TRANSIT flag in source mailbox

This is a flag stored in the source mailbox before the move begins to indicate to the source store that the mailbox is in the process of being moved. If this flag is set on a mailbox, this prevents the store from allowing new mail being delivered to the mailbox or other changes from being made to the mailbox during the move. It’s because of this flag that there’s no need to disallow logons to the mailbox while the mailbox is “in transit”.

4) Create the target mailbox

The mailbox is created in the target store and some of the basic mailbox properties and folders are set prior to moving the actual data. PR_IN_TRANSIT is also set on the target mailbox so that no mail delivery or logons will happen during the move.

5) Attempt to move the mailbox

For performance reasons, the mailbox move is attempted using a fast method that attempts to move the mailbox as a single object rather than moving each message individually. This move is done by calling the MAPI function IMapiProp::CopyTo() on the mailbox object. If the number of “bad items” selected within the wizard – this defaults to three – is exceeded during the move, the mailbox move fails and the progress is rolled back.

6) If the mailbox data is moved successfully, update the directory object

Once the mailbox data is all moved to the target store, the directory entry (or directory entries if 5.5 is involved in the move) need to be updated to reflect the new mailbox location. The AD attributes are: HomeMDB, HomeMTA, and msExchHomeServerName. The 5.5 attributes are: HomeMDB and HomeMTA.

The 5.5 directory updated by this process is that of the source server. If the source server doesn’t have a 5.5 directory, this step is skipped. If the target of the move is a 5.5 server, the move mailbox process also updates the HomeMDB and HomeMTA attributes directly at the target server in case the user logs into their mailbox before directory replication can happen.

If a mailbox is moved from an Exchange 2003 server to a server running an earlier version of Exchange (5.5 or 2000), the “wireless” attributes are also stripped off at this stage. These AD attributes are: msExchOmaAdminWirelessEnable and msExchOmaAdminExtendedSettings.

If these directory updates fail, the mailbox move is rolled back. If they succeed, then the mailbox move is done and it’s considered a successful move!

7) Clean up after a successful move!

If it’s been a successful move, we’ll have made a copy of all of the mailbox data from the source to the destination store. We won’t yet have done anything to the data in the source mailbox store (just in case the move had failed, we’d need to be able to roll back easily).

After the move is deemed successful, the mailbox in the destination store is updated to ensure the PR_IN_TRANSIT flag is no longer set. This will allow the user to login to the moved mailbox and will allow new mail (including any mail that queued up diring the move) to be delivered.

The mailbox in the source store is then removed and control is returned to the wizard so that the XML report of the mailbox move can be saved and reviewed!

Are there any myths about mailbox move I should know about?

A very common misconception is that the user must be logged off their mailbox while it is moved. This is not true, since as soon as the mailbox move is begun the PR_IN_TRANSIT flag is set on the mailbox which prevents further actions from occurring within the mailbox, protecting it during the move. There are a number of registry settings some folks have used to disable logons during mailbox moves, and while preventing logons during the mailbox moves won’t hurt anything, they’re simply not necessary.

Another myth is that the MTA must be running during an Exchange 2003 mailbox move. While this was the case in Exchange 5.5 and 2000, it is NOT the case in Exchange 2003. The MTA was required during mailbox moves in earlier versions of Exchange because when the PR_IN_TRANSIT is released on the target mailbox (in step #7, above), any mail that had been queued up for the moved mailbox would be dropped into the MTA for rerouting into the (now moved) mailbox destination. In Exchange 2003, this function is no longer reliant on the MTA routing and therefore the MTA is no longer involved in mailbox moves in this manner.

What happens to mail during the mailbox move?

During the time the mailbox is being moved, we’ve set this PR_IN_TRANSIT flag on the mailbox (on both the source and the destination mailbox). This tells the store that it cannot take local delivery of any mail into the mailbox that is being moved.

As mentioned above, in earlier versions of Exchange, during the mailbox move, mail is delivered into a special folder in the mailbox store as a temporary holding place until the mail can be delivered to the proper mailbox. Once the mailbox move is done, this mail is released to the MTA for rerouting into the destination store. However, in Exchange 2003 this is no longer the case.

In Exchange 2003, when mail fails to deliver due to the mailbox being marked “ecMailboxInTransit”, the mail is dropped into the “Messages queued for Deferred Delivery” queue within SMTP. After the mailbox move is completed and PR_IN_TRANSIT is released, the mail is rerouted through Store Driver and SMTP advanced queuing so that it is directed to the appropriate destination mailbox. This all happens transparently in the background.

What about stopping the ADC during mailbox move?

In a mixed-mode Exchange environment, the ADC can interfere with intra-site mailbox moves. If the ADC is running and a conflicting change is made to the directory object, the HomeMDB attribute can be reset to the source server. If this happens, the mailbox will appear to be “split” between the source and the destination server. To avoid this situation, be sure to stop the ADC when moving mailboxes between servers in the same site (see KB.299473 for more info). Also, be sure you don’t make any changes to mailbox objects using the 5.5 Administrator GUI while you’re moving mailboxes from the Exchange System Manager on Exchange 2000/2003!

Note that this behavior does NOT affect mailbox moves cross-site. ADC does not need to be stopped while moving mailboxes between sites using the Exchange 2003 SP1 feature.

Customizing Themes in OWA - The podcast

PodCast: Customising themes in OWA, It runs for 8 minutes, is 3mb in size and demos how to change the default OWA theme in your organisation to something really different.  The blogcast is here, and the files to replicate creating the theme are here, with a sample colour theme downloadable here.  There is also some documentation here which details how you can customise OWA in your organisation.

 

What happens during the online Normal backup?

Mike Lee of the Exchange Team posted recently a good description of how a backup really works.  This is worth the read...

Note: the following applies to “streaming” backups, not to the VSS backups.

 

Exchange transaction logs track all the changes to a database. If a database crashes, no harm done, because the transaction logs always secure changes before the crash. When you start the database again, the logs have everything needed in them to get you going back up to the instant of the crash. This crash recovery mechanism is handy too after restoring from backup. When you restore from backup, the transaction logs let you "roll forward" and not lose any changes that happened since the backup, because the logs know every change that happened to the database since the backup was done.

 

Of course, since the logs record everything, they can use up a lot of disk space. Every now and then, you have to delete the old logs. The usual way this is managed is to have the online backup process take care of it. Once you have the database backed up, old logs before the backup can be removed, and they are removed automatically as part of backup. At least, that's the way it worked in Exchange 5.5. In Exchange 2000 and 2003, it's more complex.

 

In Exchange 5.5, either all databases on a server were running or all databases were down. You couldn't pick and choose which ones you wanted to mount. This made it a cinch to figure out which log files it was safe to delete. But in Exchange 2000 and 2003, you might leave a database unmounted for months without backing it up. The Exchange team had to deal with that. So the rules about removing old log files during backup had to change. Here are the new rules:

 

- If you want backup to truncate (remove) any of your old log files, then all databases in the storage group you are backing up must be running when any one of the databases in that storage group is backed up. If even one database is down, no log files will be removed, period.

 

- If you want backup to truncate (remove) old log files on disk in a reasonably prompt way, then you must backup each database in a storage group in a reasonably prompt way. Generally, the log files in each storage group will be as old as the oldest not-backed-up database in the storage group.

 

So, don't leave a database unmounted for weeks and weeks, and backup all (not just some of) your databases frequently and you'll be fine.

 

If you want to know the details about how this works, then keep reading. But you don't have to. Really, this next section is more punishment than enlightenment. But, if you're still here, here's the process that happens when you do an online backup of an Exchange 2000 or 2003 database, or multiple databases in the same backup:

 

1. The backup agent establishes communication and initializes a backup session with the Information Store service on the target Exchange server. (In Exchange 5.5, the backup session was established with the System Attendant process.)

 

2. Depending on database transactions during the backup, the transaction logging checkpoint could be "frozen" during online backup. New changes will still be accepted and written into the database files, but the checkpoint might not move again until the backup session ends. The checkpoint log is the oldest log that has not yet had all its transactions flushed to the database files. Typically, the checkpoint lags three or four log files behind the current transaction log. If the database were to crash suddenly, Exchange knows that logs older than the checkpoint are not needed for recovery. The checkpoint log and all newer log files must be replayed into the database to recover it after a crash. When a backup is restored, the situation is somewhat similar to recovering after a crash - a certain number of log files must be replayed into the database after restoration, and these log files are always backed up with each database.

 

3. The first log that must be copied to tape with the backup is recorded in the database header in the "Current Full Backup" section. This may or may not be the current checkpoint log, depending on the backup status of other databases in the storage group.  However, this will always be the checkpoint log or an older log, never a log newer than the checkpoint.

 

4. Copy of the database files to tape begins.  If you are using Exchange 2000 SP1 or earlier, a patch file is generated (database_name.PAT). The patch file holds database changes that occur during backup that are not captured in the log files. During normal operation, all changes to the database can be reconstructed or replayed from the transaction log files. But, during backup, there may be certain complex changes that affect parts of the database already backed up as well as parts of the database not yet backed up that are not captured in the log files. These changes go in the .PAT file. Improvements made in Exchange 2000 SP2 resulted in the number of such changes being so small that they could be suspended in memory during backup of a very large database over a very long period of time without ill effect. These changes are now applied to the database files after backup finishes. If used, the .PAT file is copied to tape after the database files have finished, and then is deleted. 

 

Along with capturing changes during backup that logging does not, the patch file also contains important header information used during restoration. When the patch file was dispensed with in SP2, this information was moved into the backed up database file. Now, as a database file finishes being copied to tape, Exchange adds a single extra page to the very end of it--on tape, but not on disk. Thus, a database in an online backup is always 4K larger than the same database on disk. This page is a "mini header" that records the essential information that used to be in the .PAT file header.

 

If you run Eseutil /MH on a database that has been restored from online backup but on which recovery has not yet run, you will see the "mini header" information displayed as the Patch Current Full Backup section.

 

5. The current transaction log file is forced to "roll over" and close immediately after all database files have been copied to tape. This happens regardless of how full the log is. The current log is always named Enn.log (where nn is 00, 01, 02 or 03). After a log closes, it is given a five digit hexadecimal "generation number." So Enn.log becomes EnnXXXXX.log, and the XXXXX keeps increasing by 1 as each new log is generated.

 

The reason the log is forced to roll over is that log files cannot be backed up while they are open. This log needs to be on tape, because it contains operations applicable to the database(s) that were just backed up. Therefore, the log is closed so it can be appended to the tape. You will never see a log file called Enn.log in an online backup set. Only closed, XXXXX- numbered log files are backed up.

 

6. The range of logs needed to reliably recover the backup are copied to tape. These will include at least all the logs starting from the frozen checkpoint up through the log that was just forced to close.

 

Note that if all databases are mounted in the storage group and all databases have been selected for backup, then this range of logs will only be from the checkpoint log to the highest available numbered log. But if some databases are dismounted or not all databases are being backed up, then the range of logs copied to tape may reach back before the current checkpoint. In any case, Exchange ensures that all logs needed for replay into the backed up databases will be present on tape. Exchange follows a "better safe than sorry" policy, and always backs up all logs that might possibly be needed to get the database running again after restoration.

 

7. Log files that no database in the storage group needs for recovery are truncated (deleted from disk). The headers of all the databases in a storage group keep track of last backup time for each database, and which logs were required. If any database in a storage group is dismounted, its header will not be read and Exchange will make no calculations about which log files can be safely deleted.  And this means...if any database in a storage group is dismounted while any other database is being backed up, no log files will be deleted as a result of the backup. If you want backup to remove log files from disk, you must have all databases in a storage group running while backup occurs, or nothing will ever get deleted. This is true even if you are only backing up a single database. All the other databases must be running, even if not being backed up, in order for log truncation to happen. If all databases are mounted, then Exchange will cross-reference all the headers, and figure out which log files it is really safe to delete.

 

8. The Previous Full Backup section of the database header is updated to reflect the time and log range of the backup that just completed.

IMF and the Junk E-mail folder in Outlook

As usual, Evan Dodds explains everything we need to know about IMF!

I’ve posted several IMF-related blog postings in the past — here, here, and here. In these posts, I’ve focused mainly on the gateway behavior of the IMF. This is because that’s really what the IMF is, of course… just the gateway behavior. But when your mailbox server is running Exchange 2003 or better, there’s a whole different angle to consider: Store behavior.

As I’d mentioned, through the IMF UI in ESM, you can set the Store Action Threshold. This sets a value in the AD that is read by the Information Store on Exchange 2003 (or newer) and controls the server-side Junk E-mail folder behavior. What this means is that if the mail makes it to the Store (ie – is not stopped by IMF at the gateway), it will have an SCL stamped on it. The Exchange 2003 Information Store will detect that there is an SCL value associated with the message. The store will then make a determination on whether or not the message SCL value means it needs to be moved into the mailbox’s Junk E-mail folder.

Important to note that this IMF/SCL related Store Action is totally separate from and unrelated to the Outlook client-side Junk E-mail evaluation. It’s a bit of a confusing interaction, but they really are separate. Even if you never log onto your mailbox with Outlook 2003 in cached mode and set the Junk E-mail options to be enabled, you can still have the Exchange Junk E-mail processing move messages to the Junk E-mail folder. There’s a bit more information on Outlook 2003 client-side Junk E-mail behavior in KB.842510.

Bonus: Outlook doesn’t use the IMF provided SCL values for its client-side (cached-mode only) anti-spam determination. Instead, it does its own Junk E-mail evaluation and determines whether or not to move the mail to the Junk E-mail folder based on the settings within the “Tools->Options” Junk E-mail settings.

• “Low” setting corresponds to approximately a store action threshold of 6 (ie – it will move messages with SCL or 6 or higher to the Junk E-mail folder).
• “High” setting is more aggressive and corresponds to an SCL of between 4 and 5 (ie – it will move messages with SCL greater than somewhere between 4 and 5 to the Junk E-mail folder). Note that there’s not a 1:1 mapping between the four settings in Outlook Junk E-mail behavior and IMF SCLs, so that’s why impacted SCL values are “somewhere between 4 and 5” for Outlook.

Back to the StoreActionThreshold, much of the confusion I see in this area is around what has to be done to get the server-side Junk email processing to work. It’s important to understand that there is actually another requirement that has to be taken into account here. It’s not just the server realizing that the message processed by IMF has a particular SCL and moving it to the Junk E-mail folder… there actually has to be a thing called an “Extended Rule” in place within each mailbox to make this move happen. The Extended Rule works quite a bit like the regular mail rules you can create in Outlook against an Exchange mailbox (things like “if this message is from Tom, move it to the ‘Mail from Tom’ folder”, etc). The principal difference is that the Extended rules don’t get put into the same place as the regular Outlook rules — thus there is no 32KB size limit on Extended rules, and Extended rules don’t eat into the 32KB space set aside for Outlook rules. Also, Extended rules are used for very different things than standard Outlook rules. The Extended rule is made up of information about your junk email settings, safe senders, etc. 

Now we’ve established that in order for the Store Action to work properly, you need to have both Exchange 2003 (or better) Information Store that knows to read the StoreActionThreshold value from the AD and ALSO an Extended rule set in the mailbox to tell the store how to process the Junk E-mail. That’s where most people run into trouble, and why sometimes it seems to work and sometimes it doesn’t. It’s all about how this Extended rule gets set (or doesn’t)!

• If you fire up Outlook 2003 in cached mode, it will set this Extended rule in your mailbox automatically.
• If you fire up Outlook 2003 in online mode, it will NOT automatically set this Extended rule, although you can go into the Tools->Options dialog and change the Junk E-mail configuration to ensure it is set.
• If you fire up Outlook Web Access (OWA) 2003 and go into the Options page, you can check the checkbox to “Filter Junk E-mail”, which creates the Extended rule.

So, in short, if you’re not having all of your users login to the server with Outlook 2003 in cached mode already, the rule probably isn’t already created and won’t be automatically created just because you implement IMF.

What to do? If you’re not using Outlook 2003 or if you only have a handful of users affected by this behavior, the best option is probably to go into the affected mailboxes (or have the end-user do it) with OWA2003 and set the Filter Junk E-mail checkbox. This will set the Extended rule in the mailbox, and Junk E-mail folder will begin to function.

But what if you have a BUNCH of mailboxes to set this on? Unfortunately, there’s no built-in way to bulk set this rule on each mailbox directly. But one of my Microsoft colleagues (Jerry Wang) has put together a VBScript that will automate the process of spinning through the mailbox in OWA and setting up the Extended rule. This might be useful if you’ve a need to set this on a large number of mailboxes. This script comes with no warranty, etc and please review it before you use it, and of course you use it at your own risk! Be sure to replace the SERVERNAME, DOMAIN\Administrator, and PASSWORD with your proper information.

You feed this script a file containing a list of mailboxes (alias or SMTP address), one per line. If you can’t connect to the mailbox OWA with http://<servername>/exchange/<line-from-the-input-file>/ it won’t work, so construct the input file accordingly and change the strURL value in the script if you need to adjust the URL.

'=======================================================================
'Microsoft provides programming examples for illustration only,
'without warranty either expressed or implied, including, but
'not limited to, the implied warranties of merchantability
'and/or fitness for a particular purpose. This article assumes
'that you are familiar with the programming language being
'demonstrated and the tools used to create and debug procedures.
'Microsoft support professionals can help explain the functionality
'of a particular procedure, but they will not modify these examples
'to provide added functionality or construct procedures to meet your
'specific needs. If you have limited programming experience, you may
'want to contact a Microsoft Certified Partner or the Microsoft fee-based
'consulting line at (800) 936-5200. For more information about Microsoft
'Certified Partners, please visit the following Microsoft Web site:
'http://www.microsoft.com/partner/referral/
'=======================================================================

Option Explicit
'==========================================
Const ServerName = "SERVERNAME"
Const AdminUserName = "DOMAIN\Administrator"
Const Password = "PASSWORD"

Dim oArgs
Set oArgs = Wscript.Arguments
If (oArgs.Count < 1) Then
 Wscript.Echo "No parameter specified"
 Usage()
End If

Dim oFileObject, oLog
Dim strLine,strURL

Set oFileObject = CreateObject("Scripting.FileSystemObject")
Set oLog = oFileObject.OpenTextFile(oArgs(0),1)
Do Until oLog.AtEndOfStream
 strLine = oLog.ReadLine
 strLine = Trim(strLine)
 strURL = "HTTP://" & ServerName & "/Exchange/" & strLine
 ForceJunkEmailFolder strURL, AdminUserName, Password
Loop
oLog.Close

Set oLog = Nothing
Set oFileObject = Nothing

'==========================================
' Print the usage
'==========================================
Public Sub Usage()
 WScript.Echo "Usage:"
 WScript.Echo "Force creation Junk Email Folder by logon the mailbox by OWA."
 WScript.Echo "  cscript owalogon.vbs <file name>"
 WScript.Quit 0
End Sub

'==========================================
'Logon to specified mailbox by OWA
'==========================================
Public Sub ForceJunkEmailFolder(strURL,strUserName,strPassword)
On Error Resume Next
Dim XmlHttp
Set XmlHttp = CreateObject("Microsoft.XMLHTTP")
XmlHttp.Open "GET", strURL,false,strUserName,strPassword
XmlHttp.SetRequestHeader "Accept-Language","zn-ch"
XmlHttp.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
XmlHttp.SetRequestHeader "Cache-Control","no-cache"
XmlHttp.SetRequestHeader "Accept", "*.*"
XmlHttp.Send
If Err <> 0 Then
 Wscript.Echo "Force Create Junk Email Folder at URL " & strURL & " Failed!" 
 Exit Sub
End If

If XmlHttp.Status <> 200 Then
 Wscript.Echo "Force Create Junk Email Folder at URL " & strURL & " Failed!"
End If
Set XmlHttp = nothing
End Sub

How to use Mdbvu32.exe to identify and remove a delegate rule from a mailbox in Exchange Server

I recently ran into this exact issue and found this KB312433 article to resolve it.

SUMMARY

In Outlook clients, a damaged or corrupted delegate for a Microsoft Exchange Server 5.5, Microsoft Exchange 2000 Server, or Microsoft Exchange Server 2003 mailbox could cause the following conditions:

•	Meeting requests are incorrectly sent to previously removed delegates.
•	Non-delivery reports (NDRs) are generated when users send meeting requests to a mailbox because that mailbox has a delegate whose mailbox has been deleted.
•	You cannot add a user as a delegate if the user was previously a delegate but had been removed from the list of delegates.

This article discusses two methods to resolve this issue.

 

Legacydn.exe: great tool idea, but only in a Lab???

In a MS KB324606 Article posted yesterday, Microsoft shows us a new tool, but don't try to use it in production if there are more than one DC.

 

Summary:

Legacydn.exe is a tool that you can use to perform the following tasks:

•	Change the Microsoft Exchange 2000 Server or Microsoft Exchange Server 2003 organization name.
•	Change the administrative group names.
•	View the legacyExchangeDN values.
•	Change the legacyExchangeDN values.

Note Microsoft only supports using the Legacydn tool in a lab environment. The tool works only in a single domain controller environment.

Update for Outlook 2003 Junk Email Filter feb 2005 (KB891067)

This optional update provides the Junk E-mail Filter in Microsoft Office Outlook 2003 with a more current definition of which e-mail messages should be considered junk e-mail. This update was released February 8, 2005. You can get specific information about this update in the Microsoft Knowledge Base article Description of the Update for Outlook 2003 Junk Email Filter (KB891067

MS to acquire Sybari Software

An email I recieved today shows Microsoft intent on Buying Sybari.  I don't know about anyone else, but I have never had any good results using Sybari AV with Exchange. 

---

From: 
Sent: Tuesday, February 08, 2005 9:08 AM
Subject: Premier - INFORMATION ABOUT ACQUISITION OF SYBARI SOFTWARE COMPANY

As part of a more comprehensive approach to help secure customers, Microsoft has announced its intention to acquire Sybari Software, Inc., a privately-held company based in East Northport, NY.  Sybari security products help over 10,000 businesses protect their messaging and collaboration servers from malicious software.  These products are tightly integrated with the servers they protect.  By continuing to invest in these products, Microsoft will be addressing a critical need for businesses – better protection against malicious software.

Sybari’s antivirus products utilize third party virus scan engines to provide businesses with comprehensive protection against the threat of viruses, worms and spam.   With a layered defense strategy inherent in the product, Sybari’s Antigen reduces the window of vulnerability, provides customers with the ability to monitor, control and manage how viruses are being scanned within the network and delivers maximum protection with minimal impact on performance for messaging and collaboration servers.   

This acquisition shows Microsoft’s continued support of the partner ecosystem. Because the Sybari antivirus solution enables the use and management of multiple scan engines for virus scanning, customers get improved virus detection rates.   Sybari’s products use leading third-party scan engines from Norman Data Defense Systems, Virus Buster, Sophos, Authentium, AhnLab, Computer Associates, and Kaspersky Labs.  Microsoft will continue to work with these leading vendors and expects these partnerships to remain strong.

This acquisition is part of Microsoft’s comprehensive approach for tackling the security issues of our business customers. We intend to continue investing in solutions to protect customers against all types of malicious software, including viruses, malware, spyware, and other emerging threats.

Microsoft has posted more information on this announcement on our website at this location:

http://www.microsoft.com/presspass/press/2005/feb05/02-08SybariPR.asp

 

Updated IMF filter released today!

Hot off the presses, there’s a filter update for the Exchange Intelligent Message Filter that released today. You can download this updated filter at: http://www.microsoft.com/downloads/details.aspx?familyid=c1ea8cf1-48c9-4e43-a4eb-82d9a83fd4a7&displaylang=en. See the Readme for more info.