Log Parser Basics... Ok, a little more than basics.

Mark Minasi's Windows Networking Tech Page newsletter this month posted an article to describe details behind using the Microsoft LogParser utility.  It is good reading and will come in handy...

 

Introducing Log Parser, a Tool You Must Learn

I'm too cheap to pay for one of those Web site analysis tools, but I'd like to be able to extract a few statistics from my Web logs -- in particular, it'd be great to know how many hits a particular page had, or how many of you took a moment and read this newsletter.  When I asked my friend and IIS expert Brett Hill, he got this mystical look in his eyes -- you know the way people look when they're about to tell you about the Secrets Of The Universe that they've recently discovered? -- and beatifically intoned, "Log Parser."

Now, I'd already heard about Log Parser, but I'd also heard that it was a [fill in your favorite frustration-related adjective] nightmare to understand syntax-wise.  Brett said no problem, he was going to do a talk about Log Parser at the next Windows Connections conference.  But Brett got hired away by Microsoft -- he's now an IIS 7.0 Evangelist and yes, I did mean to type "7.0" rather than "6.0" -- and so Randy Franklin Smith, a big-time security techie, stepped in.  In his presentation, Randy did just what I needed him to do:  give me a bunch of already-working examples of Log Parser syntax so I could get started.  So I've been working with it and in this article, I'll explain why you really want to learn it and then I'll explain some of its truly nightmarish syntax.

I strongly recommend that you give this article a look.  This is a very useful tool and, of course, the price is right.

What Log Parser Can Work On

Log Parser is a free command-line tool that can analyze and report on a variety of files.  I've already suggested one use, to count the number of times that this newsletter has been viewed.  But Log Parser can also analyze event logs, your Active Directory ("show me all of the managers -- that is, someone who appears in the 'manager' attribute of my user accounts -- and compute how many people each person manages"), the file system ("show me the names of the ten largest files on the hard disk"), any text file ("how many times does the word "really" appear in this document?"), Network Monitor output, the Registry ("how many REG_DWORD values exist in my Registry?") and a number of other formats.  It will then output that data as text, a file, a datagrid, new entries in a SQL database, SYSLOG entries, XML, and so on.

Installing Log Parser and Some Sample Data

You can find Log Parser at Microsoft's downloads section.  It's a simple MSI file and so a snap to install.  Unfortunately it installs itself to a folder in Program Files without modifying the system's PATH environment variable, meaning that you've got to be in Log Parser's directory to run it or you'll get a "bad command or file name" error.  Either add Log Parser's directory to your PATH variable, or if you're lazy like me then just copy the logparser.exe file into System32.  Then open up a command line and you're ready to start parsing.

But we'll need some data to analyze.  I've simplified (and sanitized) two day's logs from my Web site and put them at

http://www.minasi.com/testlogs.zip

Right-click that URL from Internet Explorer and choose "Save target as..." to save the file to your hard disk.  Unzip it and you'll find two log files -- put them in a directory named c:\logfiles.  Make that your default directory ("cd \logfiles") in your command prompt window and your commands can be shorter, as you won't have to type "c:\logfiles" in your Log Parser commands.  With that, we're ready to go.

Note:  here's another strong recommendation:  grab that ZIP file, download Log Parser and try this stuff out.  If you're feeling lazy, remember you can always just copy a Log Parser line from the Web page you're reading and paste it into your command prompt.

A First Query

Let's try out just about the simplest Log Parser command possible:

logparser "select * from *.log" -i:iisw3c

Picked apart, it is the command "logparser," followed by a SQL query statement -- don't run away, I'll show you all the SQL you'll need today! -- followed by the -i option, which explains to Log Parser what kind of file it is (an IIS log file, in this case).  The SQL query is "Select * from *.log", which just means "get everything" -- the asterisk works in SQL the same as it does in DOS commands, meaning "everything" -- from all of the files with the extension "log" in the current directory. 

(Aside: this is why learning Log Parser is difficult -- you're trying to learn two new things at the same time.  Half of what you're trying to learn is Log Parser's syntax, which is ugly enough all by itself.  But every Log Parser query includes a SQL query, and if you've never written SQL queries then you'll find that they're a quite wide field of syntax to master as well.  I strongly recommend taking the time to browse through the logparser.chm Help file that installs in the same directory as Log Parser.  And let me note at this point that I'm not a SQL query expert, so I may not be approaching these problems in the best way.)

Controlling Where Log Parser Puts Its Output: rtp, FROM and Data Grids

You'll get ten lines of output and then a "press a key..." prompt.  Given that there are about 6491 lines in the two logs and assuming that you want to see every line, that'd mean you'd have to press a key about 649 times... yuck.  That's where the -rtp ("records to print?") option comes in; set it to 100 and it'll only pause every 100 lines.  Set it to -1 and you'll never see a "press a key..." again:

logparser "select * from *.log" -i:iisw3c -rtp:-1

Of course, that still takes a long time and is kinda useless in a command prompt window.  We can tell Log Parser to stuff the result into a file by adding the "INTO" option.  It goes in the SQL query before the FROM part.  This takes the output and puts it in a file called OUTPUT.TXT:

logparser "select * into output.txt from *.log" -i:iisw3c -rtp:-1

Open output.txt in Notepad and you'll see that you've got all of the info from the two logs nicely collected in output.txt.  But Log Parser can output its data in other ways as well.  In particular, it can use a built-in thing that 2000, XP and 2003 contain called a "data grid."  We tell Log Parser to output to something other than its "native" format (dumping all of the junk onto the screen) with the -o: option:

logparser "select * from *.log" -i:iisw3c -rtp:-1 -o:datagrid

I guess I shouldn't be surprised given that Windows is 40 or 50 million lines of code these days, but it's always interesting to learn that there's something "new" that I already owned.  Notice the "Auto Resize" button -- click it and the columns figure out how wide they should be; very nice.

Seeing What's In an Input File

Notice what's going on here -- Log Parser used spaces, tabs or commas to separate -- "parse" is the correct phrase -- each line into particular items.  You then see in the column headers the names of those items.  For example, the IP address of the person visiting my Web site is in the c-ip field, and the file that they viewed is in the cs-uri-stem field.  Or alternatively you can ask Log Parser about the "iisw3c" format like so:

logparser -h -i:iisw3c

But what we've seen so far is really just a straight dump of the logs, no computation or analysis.  What if I just wanted to see the IP addresses of my visitors?  I'd do that by restricting the things that I SELECT:

logparser "select c-ip from c:\logfiles\*.log" -o:datagrid -i:iisw3c -rtp:-1

Doing a Little Analysis and Beautifying:  COUNT, GROUP BY and AS

Ah, a much smaller amount of data, but again no analysis.  It'd be more interesting to see how often each one visited.  Again, I modify the SQL SELECT statement. I can create a new field that reports the number of times that a given IP address appears by adding a "count(*)" variable to the SELECT statement.  COUNT does just what you'd expect it to do -- it counts records.  A super-simple example might be (note that this doesn't work, I'm just introducing the idea):

logparser "select c-ip, count(*) from *.log" -i:iisw3c -o:datagrid -rtp:-1

Now, if this did work, it'd list two columns -- each IP address and how often that IP address appears.  If something showed up 30 times you'd see it 30 times with a count of 30 next to it each time.  But, again, this doesn't work, and Log Parser says that it wants a "group by" clause.   So let's do it logparser's way and add a GROUP BY clause.

logparser "select c-ip, count(*) from *.log group by c-ip" -i:iisw3c -o:datagrid -rtp:-1

That works, and we get about 1200 entries instead of almost 6500, as the duplicates are gone.  But the data grid's column label for the c-ip count is "COUNT(ALL *)," which is not all that meaningful.  That count is the number of visits that a particular IP address made, so we'd like Log Parser to call that column something more meaningful, like for example, oh, "visits."  I can do that by adding an AS clause to the list of selected items:

logparser "select c-ip, count(*) as visits from *.log group by c-ip" -i:iisw3c -o:datagrid -rtp:-1

Sorting and Shortening:  ORDER BY, TOP, DESC and HAVING

That query's output has a better column title, but the list shows the IP addresses in no particular order. It'd be nice to have it sorted by frequency, so we add the "ORDER BY" clause (this should be typed as one line although I've  broken it up so that your browser doesn't make you scroll left and right):

logparser "select c-ip, count(*) as visits from *.log group by c-ip 
order by visits" -i:iisw3c -o:datagrid -rtp:-1

Neat; now it's easy to see that one IP address visited over 140 times.   But there are an awful lot of one-visit IP addresses; can we just see the top five visitors?  Sure, with the TOP clause (again, type as one line even though I've broken it for a happier browser experience):

logparser "select top 5 c-ip, count(*) as visits from *.log group by c-ip 
order by visits" -i:iisw3c -o:datagrid -rtp:-1

Hmmm... that showed me the "top" five, all right, but the "top" of the data grid is the low numbers, as it starts out with the one-visit IPs and ends up with the multi-time visitors at the bottom of the data grid.  How to see the bottom five?  Well, there isn't a BOTTOM clause, but we can tell it to sort descending rather than its default, ascending, by adding the DESC clause to ORDER BY.  Then the most frequently-visiting IP addresses end up at the top of the results and TOP 5 works as we'd hoped (type as one line):

logparser "select top 5 c-ip, count(*) as visits from *.log group by c-ip 
order by visits desc" -i:iisw3c -o:datagrid -rtp:-1

Or alternatively I might just want to see all of the IP addresses that have visited me more than 50 times. I can do that with the HAVING clause of the SQL query (type as one line):

logparser "select c-ip, count(*) as visits from *.log group by c-ip 
having count(*) >50 order by visits desc" -i:iisw3c -o:datagrid -rtp:-1

Notice that in that case I removed the "top 5" and added "having count(*) > 50" after the "group by."  Log Parser will squawk at you if you put them in a different order.  In SQL queries want to see their clauses in a particular order -- first the SELECT, then the things you're selecting, then the FROM, then the HAVING, then the ORDER BY.  If you're using an INTO, it goes after the things you're selecting and before the the FROM.  So for example if I wanted the output of my query to go to a text file named out2.txt, I'd type (yes, again please type as one line although it's broken):

logparser "select c-ip, count(*) as visits into out2.txt from *.log 
group by c-ip having count(*) >50 order by visits desc" -i:iisw3c -rtp:-1

Side Trip:  More Useful SQL Syntax: WHERE, LIKE and More

As long as I'm talking about the format of the SQL SELECT statement, let's take a short side-trip from questing for the perfect query (remember, I'm trying to figure out how many people saw thismonth.htm) and look at what the SELECT statement can do in a bit more detail. 

If you have a really long, ugly SQL query then you can put it in a text file and refer to it.  For example, consider that last query -- it had a huge SELECT statement.  I could type its SQL part, "select c-ip, count(c-ip) as visits from *.log group by c-ip having count(*) >50 order by visits desc" in a text file like so:

select top 5
c-ip,
count(*) as visits
from *.log
group by c-ip
having count(*) > 50
order by visits desc

I then save that in a file I'll call myquery.sql (but I could call it anything).  Then this command gets the job done:

logparser file:myquery.sql -i:iisw3c -o:datagrid -rtp:-1

But I've left out a couple more types of clauses that you can put into a Log Parser SQL query.    There are also the WHERE and USING clauses.  (By the way, Log Parser only supports a subset of honest-to-God SQL.  Thank goodness.  And USING seems not to be a standard SQL clause.)  The order of these clauses in a Select statement is:

• USING
• INTO
• FROM
• WHERE
• GROUP BY
• HAVING
• ORDER BY

And no, I haven't defined what USING does yet, that's kind of beyond the scope of this introductory article.  I just wanted to offer one place where I presented all of the SQL clauses in order for easy reference.  Let's take up WHERE next.

By saying "select c-ip, count(c-ip) from *.log...", I told Log Parser to get the c-ip data from every single line on all of the logs in my local directory, and then do its analysis on those records.  But sometimes I want to tell Log Parser not to fetch every single record, but instead a subset.  I do that with the WHERE clause.  Suppose (I know this is a stretch but I need an example that fits into our data set) I only want to see entries where the IP address starts with "194."  This WHERE clause will do that :

logparser "select top 5 c-ip, count(*) as hits from *.log where c-ip like '194.%.%.%' 
group by c-ip order by hits desc" -i:iisw3c -o:datagrid

This command includes the clause "where c-ip like '194.%.%.%' and "LIKE" means "matches a particular pattern."  Patterns can either be particular characters, like if I'd written "cs-ip like '194.44.22.91,'" or they can use the "_" and "%" wildcards.  "_" means "match exactly one character" and "%" means "match zero or more characters."  The pattern '194.%.%.%,' then -- notice that LIKE patterns are always surrounded by single quotes -- would match any IP address that started with 194, followed by a period, followed by anything (the %), followed by another period, followed by another percent, followed by a final period and a percent.  Here are a few more LIKE pattern examples:

Pattern	Examples that would match
'Mark'	Mark -- "mark" wouldn't do it, case matters
'Mark%'	Mark Minasi, Mark77, Marky, Mark
'Mark_'	Marky, MarkM; Neither Mark nor Mark Minasi would work
'Mar%k'	Mark, Maraardvark
%	any string at all, or even nothing

There is also a NOT LIKE command.

So we've seen that we can use WHERE to restrict the things that SELECT does its work on.  But doesn't HAVING do that as well?  Kind of, but not exactly.  WHERE restricts the data that SELECT looks at to do its analysis; HAVING restricts the results of that analysis.  If that's not clear, let's do another query that will make it clear, as well as giving me an excuse to do some more Log Parser examples.

Querying For the Most Popular Files

We've explored our most-visiting-IP-addresses, but recall that wasn't really what I wanted to do -- I wanted to see how often people viewed thismonth.htm.  Can you see how you'd change it so that we're not seeing the most frequently-visiting IP address, but instead to see the most-requested files?  A look at the data grid output shows that the name of a requested file -- default.asp, thismonth.htm, or the like is the field "cs-uri-stem."   Form the same query as before, but replace "c-ip" with "cs-uri-stem."  Additionally, "hits" is probably a better phrase than "visits" and "file-requested" is more meaningful to most than "cs-uri-stem" and so we end up with this query (again broken here but should be typed as one line):

logparser "select top 5 cs-uri-stem as requested-file, count(c-ip) as hits from *.log 
group by requested-file order by hits desc" -i:iisw3c -o:datagrid -rtp:-1

But let's recall that I had a particular query in mind when I got started -- how many hits did my newsletter file, thismonth.htm, get?  I could figure that out from the query that showed me the top five most-visited files, but that's kind of lame.  Instead, this query does it with a WHERE clause, reporting right to the screen (again, type as one line):

logparser "select cs-uri-stem as filename, count(*) as hits from *.log 
where filename='/thismonth.htm' group by filename" -i:iisw3c

Again, notice that WHERE clause.  SQL queries fetch some subset of the fields in a database (cs-uri-stem in this case, for example) -- Log Parser didn't grab every available field in the IIS logs, just cs-uri-stem, and so it had a smaller bunch of data to work on, which presumably would make the query run faster.  So naming particular fields in the SELECT statement instead of entering * to get all of the fields reduces the number of fields to fetch before doing some kind of analysis and reporting.  In contrast, using a WHERE clause reduces the number of records fetched.  Fewer records also means less work for the query engine, which means a faster query.

Now I can offer an example where WHERE does something similar to HAVING.  We could phrase the query this way (type as one line):

logparser "select cs-uri-stem as filename, count(*) as hits from *.log 
group by filename having filename='/thismonth.htm'" -i:iisw3c

In this second query, I told Log Parser to grab the cs-uri-stem data from every single record and do a bit of computation on it (count the frequency of each file).  Once it's done with that, then Log Parser's got the breakdown of file name frequencies for every single file every encountered.  Now, I don't want to see all of those file name frequencies, I just want the frequency for thismonth.htm.  That's what the HAVING clause does -- it says "Log Parser, you've got a huge hunk of data, but I only want you to show me a tiny bit of it." 

I figured that the first query, the one with the WHERE clause would be a bit more efficient as it says to only bother computing the hit count on records about thismonth.htm, where the second computed hit counts on every single file mentioned in the log, and then only showed thismonth.htm.  And my guess was borne out, as Log Parser reports how long it takes to do something.   And yeah, the time difference was about 0.1 second, but remember there's only two log files in our test bunch -- analyzing five and a half years' worth of logs might pay off in terms of a noticeable time difference with a more efficient query.  Of course, I could be wrong --  remember, I'm just an apprentice SQLer.  (Is that pronounced "squealer?"  I'm getting these creepy flashbacks to Ned Beatty in Deliverance for some reason.)

I should also mention that, again, I'm just scratching the surface here, but here's an even more efficient way to tally the thismonth.htms:

logparser "select count(*) from *.log where cs-uri-stem='/thismonth.htm'" -i:iisw3c

Controlling IIS Log Dates To Query From

Now, that's all pretty neat, except for one thing:  I've been running a Web site named www.minasi.com for quite a long time.  But I've only been offering these free newsletters since 1999 and if I recall right I've only used the "thismonth.htm" file name for the past two.  In addition, I'm really only interested in how many people have looked at this in, say, the past month.  How, then, do I tell Log Parser "do that query, but for heaven's sake don't read every IIS log I've got going back to the beginning of time; instead, only look at entries since Friday, 29 April 2005 at 7:00 PM.

As it turns out the particular iisw3c input type has a special option designed to do just that, as I discovered by looking in the "IISW3C Input Format Parameters" page of Log Parser help.  Just add the -mindatemod parameter, followed by the earliest time that you want the log searched in yyyy-mm-dd format.  For example, to only see the "thismonth.htm" hits since 29 April 2005 I'd type (as one line)

logparser "select count(*) from *.log where cs-uri-stem='/thismonth.htm'" 
-i:iisw3c -mindatemod 2005-04-29

Or, to include the time as well, add time as hh:mm:ss and put quotes around the date/time combination.  To see all the hits since 7 PM on the 29th of April 2005, I'd type (as one line)

logparser "select count(*) from *.log where cs-uri-stem='/thismonth.htm'" -i:iisw3c 
-mindatemod "2005-04-29 19:00:00"

One little quirk to remember is that the log files store time in Greenwich/Universal/Zulu time.  So in my case, as I live in the Eastern time zone, I'd have to ask not for 7 PM (which is Eastern Daylight Time as I write this) but instead for 11 PM Universal, as I'm four hours behind that time zone.  In the winter I'm five hours behind UTC, so 7 PM for me would be the next day's midnight.  

Log Parser Output Fun

Whew, that SQL syntax stuff can be rough going, if useful.  Let's take a break and play with some more fun stuff -- a few of the ways that Log Parser can output the results of your queries.  We've seen the "native" (simple text) and data grid outputs.  Just for the sake of Log Parser weirdness, try changing the output format to "neuroview (type as one line):"

logparser "select top 5 c-ip, count(*) as visits from *.log group by c-ip 
order by visits desc" -i:iisw3c -o:neuroview -looprows:1

Which produces a pretty much useless output... but it looks like the credits from the Matrix movies so you'll no doubt impress the crowd at your next presentation.

If you've got Office loaded on your system, then you can do some graphing too.  Try creating a bar chart instead of a data grid (type as one line):

logparser "select top 5 c-ip, count(*) as visits into chart.gif from *.log 
group by c-ip order by visits desc" -i:iisw3c -o:chart -charttype:bar3d -view:on

Again, look in Log Parser's Help for more ways to display its query results.  You might sometimes want to do a really complex query that can't be done in one SQL statement; in that case, you'd do the query in parts, where you do the initial query and save those results to some kind of file (probably a comma separated variable or CSV file), then do a second query on that file, and so on.

Querying the File System with Log Parser

Let's try a query or two on something other than an IIS log.  Wondering how many MP3 files are on your system?  Well, we've seen all of the fields in iisw3c-type input files.  Here we'll use the "fs" input type file and we can get a listing of its fields with the logparser -h -i:fs approach, or look in the quite helpful Log Parser help file.  I find that, not surprisingly, there is an attribute called "name"

logparser "select count(*) from c:\* where name like '%.mp3'" -i:fs

You'll probably get a complaint that it couldn't complete the task but that's because it can't read the System Volume Information folder -- the results are still correct.  (If you like, you can give yourself Read permissions to the System Volume Information folder, but you'll get the same results either way.  Unfortunately there is no way that I know of to say to Log Parser, "search all of C:\ except for such-and-such directory.")  Or total up how much space they're taking:

logparser "select sum(size) from c:\* where name like '%.mp3'" -i:fs

Querying Active Directory

Here's a quick Log Parser AD example.  Suppose I want to get a list of all of the first names in the company, and how many people have each first name.  The only trick you've got to know is the AD word for "first names," which (if you read Newsletter 45) you know is "givenname."  Second, the "FROM" part looks like

'ldap://yourusername:yourpassword@yourdomainname/wheretosearch'

So suppose I've got an administrative account called bigguy with password "swordfish" at bigfirm.com and I want to search the whole domain for user names.  The query would look like this (and it's an ugly one, again you'd type as one line although I split it into three on the page):

logparser "select givenname, count(givenname) as tally 
from 'ldap://bigguy:swordfish@bigfirm.com/dc=bigfirm,dc=com 
group by givenname order by tally desc" -objclass:User -o:datagrid -rtp:1 -i:ADS

More Resources

This was just a start with Log Parser.  The help's got lots of examples that are worth working through, and of course you can Google it for more.  You'll also find more links about Log Parser at www.logparser.com.  Apparently Log Parser's even got a book written about it!

Exchange Server 2003 Events and Errors

When you are looking at event in eventvwr.exe (Event Viewer) in Windows 2003 you see at the bottom of every Event the following:

For more information, see Help and Support at:
http://go.microsoft.com/fwlink/events.asp

Sometimes when you click on that link you get some good information back.  Sometimes…

Where is that information stored?  Can you get to it without having to open the Event Viewer?  Sure.

TechNet has an Events and Errors Message Center where you can choose a product that the error is associated with.  Exchange 2003 has one here:

Exchange Server 2003 Events and Errors

There you can enter the Source of the error and the Event ID.  For instance, Entering the source of MSExchangeIS and the Event ID 5000 (Shudder…) gives you a table that includes:

5000  Unable to initialize the Microsoft Exchange Information Store service. - Erro...

Click on the Event ID 5000 to see the same thing you would if you did it from the Event Viewer.  Don’t forget to expand the Related Knowledge Base articles at the bottom.

Multiple Unwanted NOTEs Addresses

No, this is not a slam on Lotus Notes.  This one is MS's fault.  (I did found Paul’s recent blog particularly hilarious, though.)

If you take a look at the following KB article you might get the idea that this fix is included in Exchange 2003 SP1. 
840668 Many Lotus Notes proxy addresses appear in the properties of a user account in a domain where Exchange Server 2003 is installed

The version of ntspxgen.dll in the article is 6.5.6980.87 and SP1 should bring everyone up to 6.5.7226.0 (at least).  And it was supposed to be included in SP1, but it didn't make it for some reason.  (We did get the SMTP fix and others for the same issue into SP1.)  So, why might you need the fix?  Well, in certain circumstances the RUS can go a little crazy and start stamping objects over and over again adding NOTES addresses again and again.  Soon you may get to a point where you have "packed pages" (Event ID 1171 with error -1026) and the replication between you 5.5 environment and your Active Directory may stop.  Once you get to this point you have to do some work to get rid of the extra proxy addresses.  See 318774 Removing duplicate and unwanted proxy addresses in Exchange more information on how to resolve it.

If you don't have SP1 already installed and need to install the fix, you can get the hot fix from some of the support options at http://support.microsoft.com/oas/default.aspx?&gprid=1773.  If you already have SP1 installed you may request the POST SP1 fix, otherwise you will get an error during install with the PRE SP1 fix mentioned above.

 

Live Communications Server 2005 Service Pack 1 for Standard and Enterprise Editions

Live Communications Server 2005 SP1 improves on the features of Live Communications Server 2005 by extending the federation model, enhancing functionality, increasing security, and improving performance and infrastructure support. These improvements include:

• Tools to enable Public IM Connectivity; the ability to add contacts, send instant messages, and share presence information with users of the three main public IM service providers MSN, AOL and Yahoo!.
• Enhanced federation, which uses DNS-SRV resolution to simplify connecting to federation partners.
• New optional spim filters for better control of unsolicited instant messages.
• Support Microsoft Office Communicator 2005.
• Support for multiple tree Active Directory forests.
• Improved server API performance.

You can get specific information about this update in the Microsoft Knowledge Base article (897690): Description of Live Communications Server 2005 Service Pack 1.

Download At Source

Or Download

Live Communications Server 2005 with Service Pack 1 Trial Versions

ISA 2004 & Exchange Server 2003

Thank you to my colleague Mike Baalman for posting this link to good information on using ISA 2004 and Exchange Server 2003 together. 

http://www.microsoft.com/isaserver/solutions/exchange.mspx

 

RUS Error

We ran into an issue today at a customer site as described below.  Come to find out that the Display Specifier lost much of it's objects and running forestprep again will resolve this.  The issue is listed below as well as the KB875300 that describes the fix.

Issue:

Yesterday, during an install of the first E2K3 server in an existing E5.5 site, we ran into an issue.  Another E55 site showed no display name in E55.  Although this display name had been updated almost 2 years earlier.  We manually pulled E55 replication around the Org and the issue appeared to resolve itself.  The install yesterday completed without errors.  No RUS was needed and none were created.

    Today, we worked on another install of the first E2K3 server in another E55 site, which completed without error as well. However, this time when he went to create a new Recipient Update Service (RUS), the error "Invalid Argument - ID 80070057 was displayed.  We have since tried to create a RUS on two other E2K3 servers using different permissions, always with the same error.  

 

Solution:

According to KB875300, re-running ForestPrep will replace the missing Display Identifiers.

Recipient Policies and pure Exchange 2000/2003 sites

Bill Long has posted another good article on Recipient Policies in a Pure AG over at EHLO.  Take a look...

Once a site has no Exchange 5.5 servers in it, that old "Highest Priority" policy based on legacyExchangeDN isn't needed anymore. As people decommission their 5.5 servers and move to pure Exchange 2000 and 2003 sites, the question on how to get rid of the old site recipient policies often arises. This is not as complicated as people often think, but it does require some thought, depending on what exactly you want to achieve.

The logic of the RUS is fairly simple, but often misunderstood. The details of the decision-making process are documented in article 328738, and all you really need to know about throwing out your old policies is contained in that article. The basic idea is this:

- In the Recipient Policies container, you have a set of recipient policies.
- Each policy has a priority and a filter.
- The policy for each user is the policy with the highest priority (the lower the number, the higher the priority) with a filter that matches the user.
- The recipient policy only really matters in two situations. 1) The user is new and has no proxy addresses. 2) The policy has been "applied", as described in article 328738. In these two cases, the RUS stamps addresses on people. The RUS does not normally generate new address for recipients that already have them.

Preparation

Let me restate - the normal behavior of the RUS is to leave a user's email addresses alone, regardless of whether those addresses match the policy. It is not the normal behavior for the RUS to regenerate recipient addresses to force them to match the policy. The regeneration of proxy addresses only occurs when the policy is in an "applied" state. Update Now doesn't do it, nor does a Rebuild. These two options only control the scope of users that the RUS looks at - they do not affect the decision-making process that occurs for each user. You can Update Now and Rebuild all day long, and the RUS will never change anyone's address unless you have a policy in the "applied" state. This is why article 328738 is divided into two sections - one on the type of query, and another on the decision-making process.

This means that as long as none of your policies are currently applied, reconfiguring your recipient policies has no impact. Before you start making changes that will cause users to fall under different policies, though, it's a good idea to be 100% sure that no policies are applied. To do this, use ADSI Edit or LDP.EXE to go to your Recipient Update Services container and view the properties on each Recipient Update Service. Make sure that the gatewayProxy attribute on these objects is clear. This process is described in article 821743. Also, don't confuse gatewayProxy on the RUS objects with gatewayProxy on the Recipient Policies themselves. They serve two different purposes.

GatewayProxy on the RUS should only be populated for a limited amount of time. It is populated when the policy is applied, and the RUS begins processing all the users who match the filter on the applied policy. If the RUS finishes processing these users successfully, it will clear the corresponding entries from gatewayProxy to take the policy back out of the applied state. However, for the reasons described in article 821743, this doesn't always happen, and a policy may be left in the applied state indefinitely. There was also a bug in Exchange 2000, described in article 835156, where the policy would continue applying even after gatewayProxy was cleared, until the System Attendant service was restarted. This fix was included in the latest rollup for Exchange 2000.

As long as gatewayProxy is empty on all your RUS objects, and you have the article 835156 or a later fix, the RUS will not be making changes to any addresses just because a user's addresses don't match his current policy.

If you're still nervous about making these changes, consider getting an export of everyone's email addresses before you start. This is easy to do with ldifde, using syntax like:

Ldifde -d "DC=domain,DC=com" -r "(&(mailnickname=*))" -l proxyAddresses -f proxies.txt

This command line will export the proxyAddresses attribute for every mail-enabled object in the specified domain, and you can also add any other attributes you like to the -l parameter. You can use a simple script to reformat this file as an import and put all the proxy addresses back in their previous state if something goes wrong.

Creating the new policies

This is the part that takes some thought, and the solution will vary from one organization to another. If all your users should have a single addressing scheme, just configure the Default Recipient Policy accordingly and you're done. If you need several different addressing schemes, determining the appropriate filters for your new policies may take some planning. You'll need to identify an attribute on the users that distinguishes one group of people from another. Are your email addresses based on location? How about a filter based on city (the "l" attribute), or state (the "st" attribute). Are your email addresses based on department? Then maybe a filter based on that (the "department" attribute) would work. If none of the existing attributes on the users will work for you, you can always use one of the extension attributes, such as extensionAttribute1. The ADModify tool can be handy when you need to populate an arbitrary attribute on a large number of users. Once you know what attribute you want, building a basic filter using that attribute is easy:

(&(mailnickname=*)(myAttribute=myValue))

This filter matches any mail-enabled object which has the corresponding attribute value you've specified. For simple filters like this, you may find it easier to just choose Custom Search, click the Advanced tab, and type the filter in manually, instead of trying to choose all the appropriate checkboxes to do what you want.

Once you have configured your new policy, click OK, but beware! At this point, if you have changed the e-mail addresses that were specified on the policy by default, ESM will prompt you asking if you want to apply the policy. Clicking Yes will populate gatewayProxy and could cause the regeneration of email addresses on anyone who falls under this policy. It is best to choose No unless you want the RUS to go around changing addresses on people.

Deleting the mixed-site policies

Once you have your new policies done, simply use Exchange System Manager to delete the unwanted 5.5 site policies. Of course, you can do this step first if you like. Unless you've already created other policies that will match the users in those sites, they will now fall under the Default Policy. But this doesn't matter - since the Default Policy is not in the applied state (you know this because you checked gatewayProxy), their addresses will not be regenerated to match it.

Verification

After your old legacyExchangeDN-based policy is deleted and your new policies are configured, you're done. If you want to verify your policy configuration, you can run a Rebuild. As the RUS processes each user, it will update their msExchPoliciesIncluded to reflect the objectGUID of the policy they fall under (though, again, it will not update their proxyAddresses to match this policy as long as you have not applied the policy). You can look at the GUIDs stamped in msExchPoliciesIncluded to verify that the users are getting the policies you intended. However, keep in mind that a Rebuild can take hours or even days in the largest environments, so carefully consider this before running a Rebuild. Another option is to just make changes to a few select users. Change their Description, Zip, or anything you like. The RUS will evaluate them and stamp the new msExchPoliciesIncluded value.

As a general rule, it is not a good idea to have users fall under a policy that their addresses don't match. Someday, someone is going to apply your policies if only by accident or as a troubleshooting measure. If you want to manually configure email addresses on users and never have them be affected by applying a policy, it's best to go to the Email Addresses tab and clear the "Automatically update e-mail addresses based on recipient policy" checkbox. Once you do this, the RUS won't touch that user's email addresses even if you apply the policy. This is another operation that can be automated for a large number of users by using ADModify.

Enable Remote Desktop Remotely!

Have you ever needed to connect to a Windows Server 2003 machine only to find out that Remote Desktop or TS is not configured or enabled?  Mitch Tulloch has a solution that is shown in detail at Windows Server Hacks: Remotely Enable Remote Desktop.

Remote Desktop is a cool feature of Windows Server 2003 that lets you remotely log on to and work at a machine as if you were seated at the local console (in Windows 2000 Advanced Server, this feature was called Terminal Services in Remote Administration Mode). Remote Desktop can be a lifesaver for fixing problems on servers at remote sites, but what if you forgot to enable the feature before you shipped the server out to Kalamazoo? Enabling Remote Desktop is easy if the server is in front of you: just log on as an administrator, open System in Control Panel, select the Remote tab, and under Remote Desktop select the checkbox labeled "Allow users to connect remotely to this computer." Unfortunately, you can't use the System utility to enable Remote Desktop on a remote machine, though you can access some properties pages of System using Computer Management by first connecting the console to a remote computer, then right-clicking on the root node and selecting Properties. Unfortunately...

Scripting Exchange Part Two

MSExchange.org - Scripting Exchange Using VBScript and ADSI (Part 2)

"The first part of my scripting series discussed ways of accessing and searching for Exchange objects such as users and contacts in Active Directory. This second part of the series will go over creation of a new object, and over the most important attribute of an Exchange recipient, the e-mail address."

 

Exchange 2003 Utilities Updated

A few of the Exchange Server 2003 utilities that Microsoft provides have been updated and published to the downloads area of their site.

Exchange Server 2003 Jetstress Tool

Exchange Server AUTD Binding Cleanup

Exchange Server Load Simulator 2003 (LoadSim)

Exchange Server MSSearch Administration Tool

Update: The reference to NAS support on the Jetstress download page has been removed.  Thanks to Matt Lathrum.

 

Exchange Server User Monitor Tool

Microsoft has released another utility for our toolbox: the Exchange Server User Monitor Tool

It must be installed on the Exchange Server you want to monitor users for and requires Exchange 2000 sp2 (or later) or Exchange 2003 sp1 (or later).

You can read more about ExMon at the Exchange Team blog and of course in the documentation.  Even though it has been released as an installer package, it is really just a self extracting file.

William Lefkovics

Updated Exchange MP Configuration Wizard

For those using MOM 2005 to manage Exchange, note that Chris Harris of Microsoft advises that a new version of the Exchange Management Pack Configuration Wizard has been released.

The update fix list includes:

1.  Crash due incorrect resource string when there is an error setting the messageTrackingEnabled property on the AD (Error msg: Index (zero based) must be greater than or equal to zero and less than the size of the argument list.)  This update means you no longer have to apply the .NET update for the Wizard to properly enable Message Tracking.

2.  A valid domain is reported as not valid for the Mailbox Access Account if domain uses DNS in other forest (Error msg: "Invalid domain for the mailbox access account")

3.  DCR: Wizard does not support NetBIOS domain names containing a period (.)

4.  Wizard incorrectly matches a valid list of services to monitor if more than one service is inside double-quotes

5.  Wizard incorrectly assumes that the AD schema has been extended for Exchange 2003 when configuring Exchange 2000 servers

 

IBM In Denial Over Lotus Notes

The marketing folks in IBM's Lotus division are starting to sound like the Black Knight in Monty Python and the Holy Grail, who insists he's winning a fight even as he loses both arms and legs: "'Tis but a scratch," the Black Knight declares after one arm is lopped off. "Just a flesh wound," he says after losing the other. "I'm invincible!"

The same goes for IBM's Lotus, which keeps declaring victory even as Microsoft carves it up. First Microsoft consumed Lotus's 1-2-3 spreadsheet business. Lotus spinmeisters insisted Microsoft wasn't really winning because Lotus 1-2-3 still had a larger installed base. Eventually that wasn't true either. Now Microsoft's Exchange has clawed its way to the top of the corporate e-mail market, displacing Notes/Domino, which once dominated e-mail and was the main reason IBM paid $3.2 billion to acquire Lotus in 1995.

Exchange, first released in 1996, now outsells Notes/Domino and has a larger installed base and more momentum, many analysts say.
Yet IBM still claims Notes is the "best-selling" e-mail product on the market. And the rah-rah Lotus faithful--consultants who make a living by maintaining Notes and writing specialized Notes applications--promote this version of reality to their customers. (IBM declined to comment for this article.)

Conceived in 1984 and introduced in 1989, Notes has a user interface that some consider dated and overly complex. The product is also costly to operate, some say. Even IBM seems to think something new is needed. It has developed an e-mail program called Workplace Messaging, which is part of a new family of software products. IBM says Workplace Messaging won't replace Notes. Instead, IBM says Notes will, ahem, evolve and become part of the Workplace family. The truth is that the spin is aimed at keeping Notes customers from dropping Notes and switching to something else.

Meanwhile, Notes consultants have resorted to bashing market researchers who say Notes is slipping, suggesting on blogs that these analysts are extreme outliers who lack credibility and/or are shills who were paid off by Microsoft. But the fact is that all but one of the top market research firms say Microsoft Exchange is now the leading e-mail product. Even Gartner Group, the lone holdout, says IBM maintained a mere 1.8% market share lead--but that was in 2003.

Nevertheless Notes zealots cling to this Gartner statistic, touting it to support their "We're number one" rhetoric.

For the record, here is a rundown of analyst opinions:

• Gartner: In 2003 Gartner estimates IBM had a 46% share of e-mail sales vs. 44.2% for Microsoft. The firm won't comment on 2004 sales yet.
• IDC: In 2003 Microsoft outsold IBM $770 million v. $709 million. The final 2004 figures have not been tallied but the preliminary estimate is that "Microsoft will go up and IBM will go down. The delta is growing," analyst Mark Levitt says.
• Ferris Research: Microsoft has a 60% share of the business e-mail market vs. 25% for Lotus. Microsoft has a larger installed base and generates greater license fees than IBM.
• Meta Group: "Exchange is picking up share, and Notes/Domino is losing share. I'm seeing more defections from Domino. There is migration from Domino to Exchange," says analyst Matt Cain.
• Radicati Group: In 2004, Microsoft had 115 million seats installed worldwide vs. 83 million for IBM. By 2009, Microsoft will have 200 million seats installed vs. 103 million seats for IBM's Notes and Workplace products combined.
• Info-Tech Research Group. Among midsize companies ($1 billion or less in annual sales) in 2004, Microsoft had a 33% market share and IBM had 25%. By end of 2005, Microsoft will reach 35% with IBM dropping to 18%. "A lot of Notes shops are looking elsewhere," analyst Carmi Levy says.

Despite all this research, IBM and its head-in-the-sand Lotus "community" insist they're still number one. Which, paradoxically, helps explain why they're not.

Future version of LCS2005 client for MS mobile platform

Looks like Microsoft is creating an IM client for a future version of Windows Mobile that will work with LCS 2005, according to Ed Simnett, group product manager at MS.  However, RIM is working on the same type of client and may beat Microsoft to the punch.

Full story here: http://news.zdnet.com/2100-3513_22-5675587.html?tag=nl.e539

How Mailbox Manager processes recipients

Here is a good post from Mike Lagase on Mailbox Manager details...

This is a part 2 of my 3 part series on Mailbox Manager, please go here to read the part 1!

There is a task (CMBCleanTask) within MAD.EXE that runs every 15 minutes and figures out if the schedule information says it is now time to run. The schedule is controlled by the attributes msExchMailboxManagerActivationStyle and msExchMailboxManagerActivationSchedule on the server object in the DS.

The values for 'style' are:

         0 = Never
         1 = Schedule
         2 = Always
         3 = Custom

If we determine that we need to run, MAD starts the Clean Mailbox function to do the actual work. When this function is called, we first look in each of the databases on the server for mailboxes.  Based on the mailboxes this process found, it then queries Active Directory to see what user has a Mailbox Manager Policy applied to them. Once the user's policies have been identified, we then, based on how that policy is configured, go in to each mailbox and clean them accordingly. Once a mailbox is processed, an email notification is sent to the end user, if configured. At the end of the process, we send the administrator report, whether it is a detailed or a summary report. This report beginning with Exchange 2000 SP3 and later is sent via an authenticated MAPI logon using LocalSystem. Versions prior to Exchange 2000 SP3 used CDO for sending this report and needed anonymous authentication on the SMTP Virtual Server to relay properly.

How do we calculate if we process a message?

This calculation is determined by the following three MAPI properties on a message. If any one of these properties is less than the limit set by the Mailbox Manager rules, then we will skip that message.

- PR_MESSAGE_DELIVERY_TIME
- PR_CLIENT_SUBMIT_TIME
- PR_LAST_MODIFICATION_TIME

For a message to be moved or deleted by Mailbox Manager, all three of the above properties need to exceed the limit that has been set on the policy. So if you have a policy that states that any message in your Inbox older than 30 days needs to get cleaned by the Mailbox Manager process, we check the above properties and if all are greater than 30 days, we go ahead and clean that message. For other message classes such as Appointments, Task, Journal items, etc., we perform additional checks and look at different MAPI properties during this cleaning process. More information regarding those other checks can be found at http://support.microsoft.com/?id=302804

When Mailbox Manager first came out back in Exchange 5.5, we based the above calculation different than what we do in Exchange 2000 or Exchange 2003. We used to base this on the PR_MESSAGE_DELIVERY_TIME of a message only. There are some instances where you would like to base the cleaning on this property alone. This can be modified by changing the value for msExchMailboxManagerAgeLimit on the Mailbox Manager policy to a value of 3. All possible modifications to this value can be found here.

Below is a listing of attributes associated with Mailbox Manager Policies:

Policy Attributes
msExchMailboxManagerAgeLimit
msExchMailboxManagerCustomMessage
msExchMailboxManagerFolderSettings
msExchMailboxManagerKeepMessageClasses
msExchMailboxManagerMode
msExchMailboxManagerSendUserNotificationMail
msExchMailboxManagerSizeLimit
msExchMailboxManagerSizeLimitEnabled
msExchMailboxManagerUserMessageBody
msExchMailboxManagerUserMessageFooter
msExchMailboxManagerUserMessageHeader

Server Attributes
msExchMailboxManagerActivationStyle
msExchMailboxManagerActivationSchedule
msExchMailboxManagerMode
msExchMailboxManagerReportRecipient

MS05-021: Vulnerability in Exchange Server Could Allow Remote Code Execution (894549)

This update resolves a newly-discovered, privately-reported vulnerability in Microsoft Exchange Server that could allow an attacker to run arbitrary code on the system. The vulnerability is documented in the "Vulnerability Details" section of this bulletin. An attacker who successfully exploited this vulnerability could take complete control of an affected system. An attacker could then install programs; view, change, or delete data; or create new accounts with full user rights.
[Microsoft Security Bulletins]

The new Windows Server 2003 TechCenter

New: Windows Server 2003 TechCenter The Windows Server 2003 TechCenter offers a key step toward improving the documentation experience by enabling authors to update content based on customer feedback. It presents views into documentation by language, technology, task (or documentation category), and documentation set. The TechCenter hosts Microsoft Windows Server 2003 technical documentation and updates (including the new Service Pack 1).

Exchange User Monitor (Exmon) released

Introducing the Microsft Exchange User Monitor (Exmon) tool

Chris Mitchell has announced that this month's Exchange Web release contains a tool that the Exchange performance, development, and operations teams at Microsoft have used for quite some time called Exchange User Monitor (Exmon) and can be downloaded here.  Exmon for the first time allows an Exchange administrator the ability to see in amazing detail the performance of an Exchange server.  Shown on a user by user basis, Exmon allows you to see how much CPU, latency, network traffic, and disk each user on an Exchange server consumes.  It can be run in almost realtime (minute by minute analysis) or over longer (multiple-hour) capture periods.  Exmon also 'bubbles' up data sent back to the Exchange server from Outlook 2003 and higher about the user's actual experience, showing the actual RPC (network+server) latency and even the name of the process talking to the Exchange server (so you can see ActiveSync usage and other 3rd party MAPI applications).  The data Exmon exposes is the 'raw' data that many of the Exchange Performance counters use in calculating the running averages. 

Internally, this tool was used to help understand the performance of Outlook 2003 and other MAPI applications during the development of Exchange Server 2003.  We use it to understand the broad impact of performance across a server, but also to troubleshoot specific performance problems with individual users.  The impact to the server being 'traced' is minimal, allowing it to be run on very large servers. 

On a personal note, I recommend looking at a number of traces to 'aquaint' yourself with what's normal.  Exmon data can be highly dynamic in range.  It's best to use longer traces rather than smaller ones, because MAPI traffic can be very bursty, unless you're trying to look at a very microscopic level.

 

Network Monitor 3.0 beta soon

According to Neil Leslie, general manager of Microsoft Corp.'s customer service and support group, the company within six months will release a beta version of Network Monitor 3.0, an upgrade of a tool that has shipped as part of its Systems Management Server (SMS) software. What will be different in the next SMS release, Leslie says, is that Netmon won't have a "90-day time bomb" that turns off the tool unless you buy it. In other words, if you get SMS, you'll get Netmon 3.0. Free. Netmon captures and stores network packets for analysis. It can filter packets by protocol type and let you find devices on your network and track their packet-broadcasting rates. The 3.0 release adds a Visual Basic-like scripting language so you can easily customize it, says Leslie. Today, he notes, you need C and assembler language skills to do so.

Leslie says Microsoft will also make available later this year D-Code, its database of the various service and support tools that the company uses internally. The database not only lists what's what, but it also rates the effectiveness of what's what. Leslie says he wants other companies to rate their troubleshooting and analysis tools inside D-Code so the info can be shared broadly.

Exchange Server due out in 2006

"Andy Lees, corporate vice president of marketing for Microsoft's server and tools business, revealed the ship date Tuesday."

This suggests that Exchange 12 will arrive sooner versus later.  It was previous suggested the release date might not be until early 2007.

Lees also indicated that Exchange 12 will support "both the 64-bit extensions and the dual-core technology."  I don't think that includes Itanium.

Exchange 2003/Windows 2003 sp1 Cluster Patch

This update only applies to Exchange Server 2003 clusters running on Windows Server 2003 with the recently released Windows 2003 sp1.  MS KB 841561 fixes "'500 - Internal server error' error message when a user tries to access a clustered Exchange Server 2003 back-end server by using Outlook Web Access."

SYMPTOMS

You install Microsoft Windows Server 2003 Service Pack 1 (SP1) on a server cluster. The server cluster is running a clustered Microsoft Exchange Server 2003 back-end server. You experience the following symptoms:

•	If a user tries to access a mailbox by using Microsoft Outlook Web Access, that user receives the following error message: 500 - Internal server error
•	If you log on to a client computer by using administrator rights and then try to access a mailbox by using Outlook Web Access, you are successful.

 

CAUSE

This problem is caused by some of the security enhancements that are included with Windows Server 2003 SP1.

In this scenario, when a user tries to access a mailbox by using Outlook Web Access, HTTP requests into the clustering API by impersonating the logged-on user. However, there have been security changes in Windows Server 2003 SP1. The security restrictions for the APIs that perform remote registry access have been changed. Therefore, the logon attempt is not successful.

Windows 2003 SP1 Security Configuration Wizard and Exchange servers

Nino Bilic has posted this more detailed information on the SCW and E2K3... 

Now that Windows 2003 SP1 is out, I wanted to mention a tool that has shipped as part of Windows 2003 SP1. While the tool itself is not installed by SP1, the shortcut to the Help file is placed on the server desktop when SP1 is installed.

What does that have to do with Exchange? About the tool:

Security Configuration Wizard (SCW) is an attack surface reduction tool that is part of Windows Server 2003 SP1.  SCW uses a roles-based metaphor (e.g. "File Server", "Web Server", "Domain Controller", etc.) to determine the desired functionality of a particular type of server, then disables functionality that is not required for the role(s) the server needs to perform.  Specifically, SCW:

    - Disables unneeded services
    - Blocks unused ports
    - Allows further (address or security) restrictions for ports that are left open
    - Prohibits unnecessary web extensions (if running IIS)
    - Reduces Protocol Exposure (SMB, LanMan, LDAP)
    - Defines an Audit Policy

SCW guides you through the process of creating, editing, applying, or rolling back a security policy based on the selected roles of the server. The security policies that are created with SCW are XML files that, when applied, configure services, network security, specific registry values, audit policy, and if applicable, Internet Information Services (IIS).

So - really, what does all this have to do with Exchange, you ask?

There is a known issue with Exchange server installed into a non-default path (something other than %ProgramFiles%\Exchsrvr) where SCW is run and application of resultant policy might cause Exchange Server not to be accessible by clients anymore. The possible gotcha is in the "Network Security" portion of SCW which configures the Windows Firewall. This portion of SCW is used to turn on and add exceptions to the Windows Firewall. Exceptions are added by pointing the Windows Firewall to the EXE file to the application that is exempt from firewall blocking. SCW however expects those applications (in our case - services) to be in their default installation paths.

Now, before we start to get nervous, we should understand that the SCW will indicate to the Administrator if it runs into this problem. In other words - the UI will indicate there are issues with services that were "not found", and the Admin will have a chance to correct this before any changes are made to server configuration.

Once you get to the SCW part that configures Network Security, and if you are running SCW on the server that is not installed into the default installation path, you will see this:

 
In above example - the MTA Stacks, System Attendant and Store are the services that SCW expects to find in one location, but the actual EXE files are not physically there. If we continued to run SCW here and said YES to the popup that will warn you that some services were not found - after the server reboot, those services would not be accessible by their clients, as Windows Firewall would block their ports.

Please see the following article for information how to configure SCW to point to valid file locations, as well as how to recover if the server is already in the situation where services are blocked by Windows Firewall after SCW policy was applied:

896742 After you run the Security Configuration Wizard in Windows Server 2003
http://support.microsoft.com/?id=896742

There are few extra things I wanted to mention on this:

At the end of the SCW run, you will save the policy under a name you can choose. SCW lets you then take this policy and apply it to other servers in your Organization.

The possible problem here is - if the policy was created on the "default installation path" Exchange server, it will cause problems after import on the "non-default installation path" Exchange server, and vice versa. The above warnings (and a chance to correct them before policy application) are seen only during the policy creation not during the policy import.

Additionally - assuming that Windows Firewall was activated on the server and exceptions were added to the firewall policy, if administrator then adds a component or a service to the server (for example, POP3, IMAP4 or SRS are enabled) - the service will effectively be blocked from it's clients until the SCW is re-run, the added service is approved through it and policy is reapplied. Alternatively, you can manually add the service into exceptions of Windows Firewall.

I personally love the tool, but - we just have to be careful when using it!

- Nino Bilic

[You Had Me At EHLO...]

E2K3 and the Security Configuration Wizard in Windows Server 2003 SP1

A new KB article has been published for Exchange 2003 when applying Windows Server 2003 Service Pack 1.  It involves servers that the install path for the Excahnge bits were changed from the default.

 

896742 After you run the Security Configuration Wizard in Windows Server 2003 SP1, Outlook users may not be able to connect to their accounts

Interesting Exchange KB articles

Here are a couple of interesting KBs that have come out or been updated recently:

KB.884863 (http://support.microsoft.com/kb/884863) – “Update to Exchange 2000 Server adds application event logging for deletion of public folders”

An additional feature that is related to public folder diagnostics logging is now available for Microsoft Exchange 2000 Server. When you use this new functionality, an event is generated in the Event Viewer Application log when a user deletes an Exchange 2000 public folder. If the Public Folders General category is set to Medium logging or higher, an event that similar to the following will be logged when a user deletes a public folder in Microsoft Exchange 2000 Server:

Event Type: Information
Event Source: MSExchangeIS Public Store
Event Category: General
Event ID: 9682
Description: Folder /Name_of_Public_Folder with folder ID 9-9999B was deleted by /o=Exchange_Organization_Name/ou=Exchange_Site_Name/cn=Recipients_Container/cn=User, user account Domain\NTAccount_Name.

 

Note, you can also get this hotfix for Exchange 2003 Post-SP1 by referencing KB.891968 with PSS.

 

 

KB.895847 (http://support.microsoft.com/kb/895847) – “Multi-site data replication support for Exchange 2003 and Exchange 2000”

 

You can use replication technology to help provide high availability for Microsoft Exchange Server 2003 or Microsoft Exchange 2000 Server data. Exchange 2003 and Exchange 2000 replication solutions are provided by third-party program vendors. This article describes our support policies for Exchange when Exchange is used together with a third-party replication solution.

 

(Translation: this is the definitive “Exchange on replicated storage” KB article. It covers synchronous replication, asynchronous replication, software replication, Geo-clusters, you name it!)

 

 

KB.175481 (http://support.microsoft.com/kb/175481) – “Exchange single-instance storage and its effect on stores when moving mailboxes”

 

This KB article about single-instance storage behavior during mailbox moves (ie – that we keep it if you use the built-in move process) has been updated to include the cross-site mixed-mode Exchange 2003 SP1 feature. The article clarifies that these cross-site moves also retain SIS, unlike the old “Exmerge” method of moving mailboxes cross-site.

Interesting Exchange KB articles

Here are a couple of interesting KBs that have come out or been updated recently:

KB.884863 (http://support.microsoft.com/kb/884863) – “Update to Exchange 2000 Server adds application event logging for deletion of public folders”

An additional feature that is related to public folder diagnostics logging is now available for Microsoft Exchange 2000 Server. When you use this new functionality, an event is generated in the Event Viewer Application log when a user deletes an Exchange 2000 public folder. If the Public Folders General category is set to Medium logging or higher, an event that similar to the following will be logged when a user deletes a public folder in Microsoft Exchange 2000 Server:

Event Type: Information
Event Source: MSExchangeIS Public Store
Event Category: General
Event ID: 9682
Description: Folder /Name_of_Public_Folder with folder ID 9-9999B was deleted by /o=Exchange_Organization_Name/ou=Exchange_Site_Name/cn=Recipients_Container/cn=User, user account Domain\NTAccount_Name.

 

Note, you can also get this hotfix for Exchange 2003 Post-SP1 by referencing KB.891968 with PSS.

 

 

KB.895847 (http://support.microsoft.com/kb/895847) – “Multi-site data replication support for Exchange 2003 and Exchange 2000”

 

You can use replication technology to help provide high availability for Microsoft Exchange Server 2003 or Microsoft Exchange 2000 Server data. Exchange 2003 and Exchange 2000 replication solutions are provided by third-party program vendors. This article describes our support policies for Exchange when Exchange is used together with a third-party replication solution.

 

(Translation: this is the definitive “Exchange on replicated storage” KB article. It covers synchronous replication, asynchronous replication, software replication, Geo-clusters, you name it!)

 

 

KB.175481 (http://support.microsoft.com/kb/175481) – “Exchange single-instance storage and its effect on stores when moving mailboxes”

 

This KB article about single-instance storage behavior during mailbox moves (ie – that we keep it if you use the built-in move process) has been updated to include the cross-site mixed-mode Exchange 2003 SP1 feature. The article clarifies that these cross-site moves also retain SIS, unlike the old “Exmerge” method of moving mailboxes cross-site.

Antivirus getting in the way of IsAlive checks

 

Evan Dodds wrote a long time ago about the IsAlive behavior in Exchange 200x clusters, but one interesting scenario was brought to his attention recently.

Remember that the IsAlive check for the SMTP resource requires the cluster node actively running the SMTP resource to be able to connect into port 25 on the virtual IP address bound to the particular SMTP virtual server. So, if anything prevents the cluster service from making this connection, the IsAlive is surely bound to fail!

With that background, know that some antivirus vendors have added an option to prevent “mass mailing worms from sending mail” (ie – they actively block access to port 25). This will prevent mail delivery to the server where this feature is enabled, whether it’s a cluster or not. Clearly it shouldn’t be enabled on an Exchange server.

But, as I mention above, if you have this option set on an Exchange 200x cluster you are going to have problems getting the SMTP resource online (and keeping it online) above and beyond any mail delivery issues you might encounter!

A Brief History of Exchange Server

 

A brief history lesson, for those of us that have lived through this, it is great to see where we have been.

Microsoft Exchange Server started its life as Exchange Server 4.0, shortly after Microsoft discovered the Internet.  Prior to Exchange, Microsoft had another email product called Microsoft Mail. The last version of MS-Mail was 3.5 and it was released in September of 1995. Microsoft hoped that by numbering the first version of Exchange as 4.0, it would convince users to migrate quickly.

Regardless of the version number, Exchange 4.0 was a version 1.0 product and it was not an upgrade to MS-Mail 3.5. To get from MS-Mail 3.5 to Exchange Server 4.0 was a full-scale migration. Exchange 4.0 also had a number of severe limitations and somewhat limited functionality (especially when compared to the products available today). However, it introduced many capabilities that are absolutely recognizable even in today’s much more mature product. Exchange Server 4.0 was followed by Exchange Server 5.0 and then by Exchange Server 5.5.

Exchange Server 5.5 was the first version of the Exchange product line to really take off. Exchange Server 5.5, especially by service pack 3, was a very usable product. It is estimated that, at this writing (very early 2005) that over 25% of Exchange customers are still using Exchange Server 5.5; even despite the fact that two major versions of Exchange have been released since the release of Exchange Server 5.5. It was, and still is, a very stable product.

Exchange Server 5.5 was the first version of Exchange to truly support “the Internet” with an Internet Mail Connector (IMC). Therefore, if companies are still using older versions of Exchange Server, then they are probably not connected to the Internet, at least for email! I doubt that very many copies of Exchange Server prior to version 5.5 are still in operation.

The releases of Exchange Server up to and including Exchange Server 5.5 all shared a common feature – a directory which was maintained by the Exchange Server itself. For the purposes of this discussion, a directory is a list of items (such as subscribers, distribution lists, contacts, public folders, routing tables, etc.) and all of the attributes (information and data) about each of those items. All of the items and their attributes were stored in a database maintained by Exchange – and this database was not part of the operating system.

In a revolutionary change, Exchange 2000 Server moved the directory into the brand new Active Directory. Starting with Exchange 2000 Server, Exchange requires Active Directory and heavily uses some of the features and functionality of Active Directory.

Since the Active Directory database was originally based on the Exchange database technology, it was a natural step.

There were other revolutionary changes in Exchange 2000 Server – among them the inclusion of a Conferencing Server (which provided additional collaboration capabilities to Exchange) as well as Instant Messaging capabilities. These changes were moved back out of Exchange Server and into a separate product (Microsoft Live Communications Server) as of Exchange Server 2003.

As a minor aside, the change in name between all prior versions of Exchange Server Version and Exchange 2000 Server was another modification that lasted for only a single product release.

The movement to Active Directory with Exchange 2000 Server also significantly changed the administrative model for Exchange. Historically, an Exchange administrator could control everything about an Exchange site – the subscribers, the mailboxes, the distribution lists, server configuration, etc. With the integration to Active Directory, however, that is no longer always the case. Exchange management is now separate from user and group management (groups include distribution groups, which were known as distribution lists in Exchange 5.5 and before).

In larger companies, this administrative split makes very good sense, as messaging and collaboration are just applications. Being an administrator of the messaging system shouldn’t provide application level administrators the full administrative control of the computer network too. For many (if not most) smaller companies, the distinction is meaningless and it often doesn’t seem to make any sense – why it is now necessary to use two programs for administration, whereas in Exchange 5.5 and earlier, everything could be done in a single program.

Thankfully, for these smaller environments the multiple administrative consoles may be merged into a single custom console providing the single point of administration that these companies are used to.

Exchange 2000 Server represented a major change to the core architecture of the Exchange product, and in some ways the integration with Active Directory again made it a version 1.0 product. Even so, Exchange 2000 was a significant improvement over Exchange 5.5. Many of the Exchange 2000 deficiencies were corrected in Exchange 2003. Exchange 2003 has proven itself to be an extremely stable and feature rich platform.

Table 1-1. Versions of Exchange Server and related products

Product Name	Release Date
Microsoft Mail 3.5	September 12, 1995
Exchange Server 4.0	June 11, 1996
Exchange Server 5.0	May 23, 1997
Exchange Server 5.5	February 3, 1998
Exchange 2000 Server	November 29, 2000
Exchange Server 2003	September 28, 2003

 

The release of Exchange Server 2003 also integrated a significant new feature set into Exchange – Mobility. Mobility provides access to Exchange features and functionality from non-traditional clients – such as Windows Mobile Clients (Pocket PC 2002 and Pocket PC 2003 with Microsoft ActiveSync) as well as any PDA client which supports cHTML (via Outlook Mobile Access or OMA).

Prior to Exchange Server 2003, the available mobility functionality was present in another server product – Microsoft Mobile Information Server (MMIS). Its functionality was somewhat less than what is present in the current version of Exchange. The releases of MMIS were as shown below.

Table 1-2. Versions of Microsoft Mobile Information Server

Product Name	Release Date
Mobile Information Server 2001	September 16, 2001
Mobile Information Server 2002	May 21, 2002

 

As of this writing (very early 2005), Exchange Server 2003 has had one service pack released, with a second service pack expected in the second quarter of this year. There have also been several “add on” feature packs released to add functionality to Exchange Server 2003 (IMF – Internet Message Filter being the chief one among them).

The next version of Exchange Server, codenamed E12, is expected in the second half of 2006.

Windows Server 2003 SP1 Is Coming. Before You Install It On Exchange Server 2003…

If you are looking to roll out Windows Server 2003 SP1 on your Exchange servers and you are doing clustering on back-end servers, we have a hotfix that you will need to install.
From: 841561 "500 - Internal server error" error message when a user tries to access a clustered Exchange Server 2003 back-end server by using Outlook Web Access
“This problem is caused by some of the security enhancements that are included with Windows Server 2003 SP1. In this scenario, when a user tries to access a mailbox by using Outlook Web Access, HTTP requests into the clustering API by impersonating the logged-on user. However, there have been security changes in Windows Server 2003 SP1. The security restrictions for the APIs that perform remote registry access have been changed. Therefore, the logon attempt is not successful.”
Right now, this is a hotfix that you need to call in to get. If it becomes available for public download the link will be in the above article. This hotfix requires that you have Exchange Server 2003 SP1. Another article will be available to address the issue if you do not have Exchange Server 2003 SP1 installed.As always… please test the hotfixes in your test lab before deploying to production servers.
posted on Tuesday, March 29, 2005 8:36 AM by Gerod Serafin

HP Enhances E-Mail Archiving Storage Array

Bolstering its information life-cycle management lineup, Hewlett-Packard on Tuesday made hardware and software enhancements to the HP StorageWorks Reference Information Storage System (RISS), an application-aware, content-based archiving array.

RISS, which shipped last May, is an integrated storage, search and retrieval appliance that originally was aimed at e-mail archiving, said Paul O'Brien, director of ILM for HP's StorageWorks Division. It uses what HP calls Smart Cells, which are connected on a storage grid and consist of a processor, storage and software. As pockets of data tied to an application, the Smart Cells can be physical or logical devices, and they can be assigned to specific users for read-and-write authorization, he said.

With the enhancements, Smart Cell capacity is now 850 Gbytes, up from the previous 400-Gbyte maximum, O'Brien said. As a result, the base capacity of a RISS increases to 1.8 Tbytes, from 1 Tbyte before. The list price for a RISS starts at $112,500, with extra 850-Gbyte mirrored Smart Cells available for $52,000.

IMF Enhancements via NEMX

SecurExchange IMF edition extends the IMF by:

• Allowing any number of thresholds to be defined with different actions for each threshold range. For instance, a SCL rating of 4-6 can be assigned "Quarantine", while a SCL rating of 7-9 could be assigned "Delete"
• Providing additional actions available to a triggered message.
• Additional actions like "reroute", "move to any sub folder", "copy to user", "send a message", and others may be assigned. Moving a message to any subfolder does not require Outlook 2003.
• Allowing different thresholds and corresponding actions amongst different user groups.
• For instance in a school environment, the "staff" group may have messages quarantined for a SCL range, while the "students" group would have those messages deleted.
• Automatically or manually allow business partner's messages to bypass Exchange's IMF through SecurExchange's Friendly Domain capability, minimizing false positives generated by the IMF
• Creating Safe Sender's lists to prevent Exchange's IMF from archiving or deleting messages on the SMTP gateway / frontend server that exceed the gateway 's IMF threshold.

Phishing And Complexities Of E-Mail Security Top List Of IT Pros' Concerns

http://www.messagingpipeline.com/159904303 Business-technology professionals want simple, fast ways to manage E-mail security, with many aiming to consolidate with one vendor.

Symbian To License Microsoft E-Mail Technology

http://www.messagingpipeline.com/159904213 Unexpected deal with rival may bolster Microsoft's bid to extend its dominance to mobile devices.

IBM Enters Anti-Spam Waters

http://www.messagingpipeline.com/159904164 New technology checks e-mail sender validity without resorting to special additions to MX or DNS records.