IMF and the Junk E-mail folder in Outlook

As usual, Evan Dodds explains everything we need to know about IMF!

I’ve posted several IMF-related blog postings in the past — here, here, and here. In these posts, I’ve focused mainly on the gateway behavior of the IMF. This is because that’s really what the IMF is, of course… just the gateway behavior. But when your mailbox server is running Exchange 2003 or better, there’s a whole different angle to consider: Store behavior.

As I’d mentioned, through the IMF UI in ESM, you can set the Store Action Threshold. This sets a value in the AD that is read by the Information Store on Exchange 2003 (or newer) and controls the server-side Junk E-mail folder behavior. What this means is that if the mail makes it to the Store (ie – is not stopped by IMF at the gateway), it will have an SCL stamped on it. The Exchange 2003 Information Store will detect that there is an SCL value associated with the message. The store will then make a determination on whether or not the message SCL value means it needs to be moved into the mailbox’s Junk E-mail folder.

Important to note that this IMF/SCL related Store Action is totally separate from and unrelated to the Outlook client-side Junk E-mail evaluation. It’s a bit of a confusing interaction, but they really are separate. Even if you never log onto your mailbox with Outlook 2003 in cached mode and set the Junk E-mail options to be enabled, you can still have the Exchange Junk E-mail processing move messages to the Junk E-mail folder. There’s a bit more information on Outlook 2003 client-side Junk E-mail behavior in KB.842510.

Bonus: Outlook doesn’t use the IMF provided SCL values for its client-side (cached-mode only) anti-spam determination. Instead, it does its own Junk E-mail evaluation and determines whether or not to move the mail to the Junk E-mail folder based on the settings within the “Tools->Options” Junk E-mail settings.

• “Low” setting corresponds to approximately a store action threshold of 6 (ie – it will move messages with SCL or 6 or higher to the Junk E-mail folder).
• “High” setting is more aggressive and corresponds to an SCL of between 4 and 5 (ie – it will move messages with SCL greater than somewhere between 4 and 5 to the Junk E-mail folder). Note that there’s not a 1:1 mapping between the four settings in Outlook Junk E-mail behavior and IMF SCLs, so that’s why impacted SCL values are “somewhere between 4 and 5” for Outlook.

Back to the StoreActionThreshold, much of the confusion I see in this area is around what has to be done to get the server-side Junk email processing to work. It’s important to understand that there is actually another requirement that has to be taken into account here. It’s not just the server realizing that the message processed by IMF has a particular SCL and moving it to the Junk E-mail folder… there actually has to be a thing called an “Extended Rule” in place within each mailbox to make this move happen. The Extended Rule works quite a bit like the regular mail rules you can create in Outlook against an Exchange mailbox (things like “if this message is from Tom, move it to the ‘Mail from Tom’ folder”, etc). The principal difference is that the Extended rules don’t get put into the same place as the regular Outlook rules — thus there is no 32KB size limit on Extended rules, and Extended rules don’t eat into the 32KB space set aside for Outlook rules. Also, Extended rules are used for very different things than standard Outlook rules. The Extended rule is made up of information about your junk email settings, safe senders, etc. 

Now we’ve established that in order for the Store Action to work properly, you need to have both Exchange 2003 (or better) Information Store that knows to read the StoreActionThreshold value from the AD and ALSO an Extended rule set in the mailbox to tell the store how to process the Junk E-mail. That’s where most people run into trouble, and why sometimes it seems to work and sometimes it doesn’t. It’s all about how this Extended rule gets set (or doesn’t)!

• If you fire up Outlook 2003 in cached mode, it will set this Extended rule in your mailbox automatically.
• If you fire up Outlook 2003 in online mode, it will NOT automatically set this Extended rule, although you can go into the Tools->Options dialog and change the Junk E-mail configuration to ensure it is set.
• If you fire up Outlook Web Access (OWA) 2003 and go into the Options page, you can check the checkbox to “Filter Junk E-mail”, which creates the Extended rule.

So, in short, if you’re not having all of your users login to the server with Outlook 2003 in cached mode already, the rule probably isn’t already created and won’t be automatically created just because you implement IMF.

What to do? If you’re not using Outlook 2003 or if you only have a handful of users affected by this behavior, the best option is probably to go into the affected mailboxes (or have the end-user do it) with OWA2003 and set the Filter Junk E-mail checkbox. This will set the Extended rule in the mailbox, and Junk E-mail folder will begin to function.

But what if you have a BUNCH of mailboxes to set this on? Unfortunately, there’s no built-in way to bulk set this rule on each mailbox directly. But one of my Microsoft colleagues (Jerry Wang) has put together a VBScript that will automate the process of spinning through the mailbox in OWA and setting up the Extended rule. This might be useful if you’ve a need to set this on a large number of mailboxes. This script comes with no warranty, etc and please review it before you use it, and of course you use it at your own risk! Be sure to replace the SERVERNAME, DOMAIN\Administrator, and PASSWORD with your proper information.

You feed this script a file containing a list of mailboxes (alias or SMTP address), one per line. If you can’t connect to the mailbox OWA with http://<servername>/exchange/<line-from-the-input-file>/ it won’t work, so construct the input file accordingly and change the strURL value in the script if you need to adjust the URL.

'=======================================================================
'Microsoft provides programming examples for illustration only,
'without warranty either expressed or implied, including, but
'not limited to, the implied warranties of merchantability
'and/or fitness for a particular purpose. This article assumes
'that you are familiar with the programming language being
'demonstrated and the tools used to create and debug procedures.
'Microsoft support professionals can help explain the functionality
'of a particular procedure, but they will not modify these examples
'to provide added functionality or construct procedures to meet your
'specific needs. If you have limited programming experience, you may
'want to contact a Microsoft Certified Partner or the Microsoft fee-based
'consulting line at (800) 936-5200. For more information about Microsoft
'Certified Partners, please visit the following Microsoft Web site:
'http://www.microsoft.com/partner/referral/
'=======================================================================

Option Explicit
'==========================================
Const ServerName = "SERVERNAME"
Const AdminUserName = "DOMAIN\Administrator"
Const Password = "PASSWORD"

Dim oArgs
Set oArgs = Wscript.Arguments
If (oArgs.Count < 1) Then
 Wscript.Echo "No parameter specified"
 Usage()
End If

Dim oFileObject, oLog
Dim strLine,strURL

Set oFileObject = CreateObject("Scripting.FileSystemObject")
Set oLog = oFileObject.OpenTextFile(oArgs(0),1)
Do Until oLog.AtEndOfStream
 strLine = oLog.ReadLine
 strLine = Trim(strLine)
 strURL = "HTTP://" & ServerName & "/Exchange/" & strLine
 ForceJunkEmailFolder strURL, AdminUserName, Password
Loop
oLog.Close

Set oLog = Nothing
Set oFileObject = Nothing

'==========================================
' Print the usage
'==========================================
Public Sub Usage()
 WScript.Echo "Usage:"
 WScript.Echo "Force creation Junk Email Folder by logon the mailbox by OWA."
 WScript.Echo "  cscript owalogon.vbs <file name>"
 WScript.Quit 0
End Sub

'==========================================
'Logon to specified mailbox by OWA
'==========================================
Public Sub ForceJunkEmailFolder(strURL,strUserName,strPassword)
On Error Resume Next
Dim XmlHttp
Set XmlHttp = CreateObject("Microsoft.XMLHTTP")
XmlHttp.Open "GET", strURL,false,strUserName,strPassword
XmlHttp.SetRequestHeader "Accept-Language","zn-ch"
XmlHttp.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
XmlHttp.SetRequestHeader "Cache-Control","no-cache"
XmlHttp.SetRequestHeader "Accept", "*.*"
XmlHttp.Send
If Err <> 0 Then
 Wscript.Echo "Force Create Junk Email Folder at URL " & strURL & " Failed!" 
 Exit Sub
End If

If XmlHttp.Status <> 200 Then
 Wscript.Echo "Force Create Junk Email Folder at URL " & strURL & " Failed!"
End If
Set XmlHttp = nothing
End Sub