Tuesday, October 26, 2004

Password Generator

This topic is a bit of a departure from the normal postings I do, but I found it useful and didn't want it to go to waste on just me.  Here is a link to a password generator I have used with good success.  Just put in your criteria and bing bong bang, out comes great passwords.  Good for use in high security areas and service accounts...

http://www.mytsoftware.com/dailyproject/PassGen/PassGen.html

Monday, October 25, 2004

Listing the file sizes of all Exchange Stores on all Exchange Servers in a Domain

Here is an interesting script that Glen Scales of Glen's Exchange Dev Blog created.  It is designed to report on the Exchange database size for all stores in an AD Domain. 

One of the things that I've found missing from the Exchange Management GUI's has been the ability to get the physical size of the database files (especially when you have multiple servers to manage). None of the Exchange API's accessible with script offer this type of information either.

I've found two methods you can use to grab this information via a script, the first is when the information store performs a backup via the Exchange Backup API one of the events it logs is event code 220 which states the size of each file before it is backed up. The other method is to connect to the file using the FSO object in VBS and grab the size of the file directly.

I've expanded this second method into a script that first queries for all the msExchPrivateMDB and msExchPulicMDB objects in Active Directory using ADSI. Then using the msExchEDBFile, msExchSLVFile and msExchOwningServer AD properties constructs the URL to be able to connect to each file remotely and report on the size of the EDB and STM files for that mail or public folder store. I've also used the homeMDBBL property which contains a list of all the mailenabled accounts within a mailstore to get the number of mailboxes for each mailstore. The script itself is designed to be run from the command line using cscript and will produce a console output for every Exchange server store for the domain its runs in.

The script itself looks like this I've posted a download-able copy up here

set conn = createobject("ADODB.Connection")
set com = createobject("ADODB.Command")
Set iAdRootDSE = GetObject("LDAP://RootDSE")
strNameingContext = iAdRootDSE.Get("configurationNamingContext")
Conn.Provider = "ADsDSOObject"
Conn.Open "ADs Provider"
mbQuery = "pfQuery = "Com.ActiveConnection = Conn
Com.CommandText = mbQuery
Set Rs = Com.Execute
Wscript.echo "Mailbox Stores"
Wscript.echo
While Not Rs.EOF
objmailstorename = "LDAP://" & Rs.Fields("distinguishedName")
set objmailstore = getObject(objmailstorename)
objmailstore.GetInfoEx Array("homeMDBBL"), 0
varReports = objmailstore.GetEx("homeMDBBL")
Set fso = CreateObject("Scripting.FileSystemObject")
edbfilespec = "\\" & mid(objmailstore.msExchOwningServer,4,instr(objmailstore.msExchOwningServer,",")-4) & "\" & left(objmailstore.msExchEDBFile,1) & "$" & mid(objmailstore.msExchEDBFile,3,len(objmailstore.msExchEDBFile)-2)
stmfilespec = "\\" & mid(objmailstore.msExchOwningServer,4,instr(objmailstore.msExchOwningServer,",")-4) & "\" & left(objmailstore.msExchSLVFile,1) & "$" & mid(objmailstore.msExchSLVFile,3,len(objmailstore.msExchSLVFile)-2)
Set efile = fso.GetFile(edbfilespec)
set sfile = fso.GetFile(stmfilespec)
edbsize = formatnumber(efile.size/1073741824,2,0,0,0)
stmsize = formatnumber(sfile.size/1073741824,2,0,0,0)
Wscript.echo Rs.Fields("name") & "# Mailboxes: " & ubound(varReports)+1 & " EDBSize(GB): " & edbsize & " STMSize(GB): " & stmsize
Rs.MoveNext

Wend
Wscript.echo
Wscript.echo "Public Folder Stores"
Wscript.echo
Com.CommandText = pfQuery
Set Rs1 = Com.Execute
While Not Rs1.EOF
objmailstorename = "LDAP://" & Rs1.Fields("distinguishedName")
set objmailstore = getObject(objmailstorename)
Set fso = CreateObject("Scripting.FileSystemObject")
edbfilespec = "\\" & mid(objmailstore.msExchOwningServer,4,instr(objmailstore.msExchOwningServer,",")-4) & "\" & left(objmailstore.msExchEDBFile,1) & "$" & mid(objmailstore.msExchEDBFile,3,len(objmailstore.msExchEDBFile)-2)
stmfilespec = "\\" & mid(objmailstore.msExchOwningServer,4,instr(objmailstore.msExchOwningServer,",")-4) & "\" & left(objmailstore.msExchSLVFile,1) & "$" & mid(objmailstore.msExchSLVFile,3,len(objmailstore.msExchSLVFile)-2)
Set efile = fso.GetFile(edbfilespec)
set sfile = fso.GetFile(stmfilespec)
edbsize = formatnumber(efile.size/1073741824,2,0,0,0)
stmsize = formatnumber(sfile.size/1073741824,2,0,0,0)
Wscript.echo Rs1.Fields("name") & " EDBSize(GB): " & edbsize & " STMSize(GB): " & stmsize
Rs1.MoveNext

Wend
Rs.Close
Rs1.Close
Conn.Close
Set Rs = Nothing
set Rs1 = Nothing
Set Com = Nothing
Set Conn = Nothing

Solution Accelerator for Exchange Consolidation and Migration

Well, looks like Microsoft has again created a good framework for documenting an Exchange 2003 deployment solution.  This will be a good starting point for providers....

The Solution Accelerator for Exchange Consolidation and Migration helps you accelerate the design, planning, and deployment of Exchange Server 2003 messaging systems deployed as upgrades to, or replacements for, your existing Exchange Server 5.5 or Exchange 2000 Server messaging systems. Business agility and efficiency depend on messaging systems deployed by organizational IT operations; to deliver the required functionality, IT operations themselves must be able to quickly adapt to changing business requirements, then implement secure, efficient solutions to meet these requirements. These solutions require not only great technologies, but also efficient and effective operations with all application and system services driven by mature processes and delivered to the business at an acceptable cost."

Thursday, October 14, 2004

A few basic concepts in disk sizing

I found another fantastic article on Disk IOPS and planning for Exchange on Nicole Allen's Blog.  This is a must read!

 

Excerpt: 

While I was at TechEd, I had a great time talking with our customers and hearing about their experiences as Exchange administrators.  One of the areas that have come up in discussion a lot has to do with planning sufficient disk throughput for the back-end exchange servers.  A Lead Program Manager in the Exchange team started pestering me to write it up in a blog, since if so many customers at TechEd had questions, it followed that their might be other Exchange administrators that would be interested as well.

So....I've written up the basics of what you need to know about disk sizing for the Exchange server.   I'm not claiming it's comprehensive (I'm not going to go into SAN technology details, for example, since they differ from vendor to vendor), but it should be enough to:

a) determine if you have sufficient disks

b) detect if you have a disk bottleneck

c) calculate the number of disk I/Os per second per user (also known as IOPS/user)

d) estimate how many disks you need for a new server, based on past user behavior.

Exchange Disk Sizing (it's not just for cluster anymore!)

 

Nicole Allen has a great post over at the Exchange Team blog with step-by-step info on disk sizing for Exchange servers. This is the same sort of information that you can find in the Optimizing Storage for Exchange 2003 whitepaper, but Nicole does a great job of making it make sense! 

In support, it's become very common to see servers sized with "400gb of storage" (designing primarily for required capacity), for instance, rather than with "6 spindles" (designing primarily for required performance). It seems like this is becoming even more common as Exchange design engineers work with their company's storage people to get space carved on the SAN. Oftentimes the Exchange people and the storage people aren't both aware of the needs of Exchange storage, and so design mistakes are made early in the process.

Here are some brief best practices I've seen listed for Exchange Storage design:
(credit to Dave Lalor for the high-level list)

1) First, size the number of disk spindles to accommodate the total number of IOPS required by the system. Size for peak load, understand your user profile, realize that disk has two important measures (capacity and performance).

2) Then, select the capacity of disk spindles to satisfy required data sizes. Total DB size = Number of users * mailbox size. Consider database overhead for maintenance activities (leave space for utilities to run). Also leave space for overhead like DB indexes, deleted items retention, etc. Don’t forget to include SMTP queues -- their random read/write IO profile is a better fit with the database drives than the transaction log drives if you choose to double them up. Also, remember that more items in the mailboxes will require more IOPS.

3) Separate data and log volumes for each Exchange storage group. Storage group has max 5 databases + one log. Recommendation is to put all 5 DB on one LUN and the transaction logs on a second LUN. Placing the databases on independent LUNs (rather than one larger LUN) can lead to difficulty meeting the 10 second quiescence window for VSS.

4) Tune storage array parameters. Some suggestions: 4kb cache page size (only if Exchange is the only thing on the array, otherwise leave it at 8kb). Maximize the write cache -- this is HUGE for Exchange performance; we're very write cache effective. Minimal (50-100mb) read cache. Enable cache watermarks. Enable read&write cached for all luns. Stripe element size of 64 blocks (32kb).

 

5) Align disk partitions to stripe sizeIf you've got a stripe element size of 32kb (see #4, just above), you'll want to make sure you've aligned your partitions to a 32kb boundary to prevent inefficient access to some of the blocks of data. Most of the SAN vendors have much better docs on this than I can provide here. :)

 

6) Cluster for high availability. Always design for Active/Passive rather than Active/Active. Use mountpoints for log and data per Storage Group or EVS to reduce number of drive letters required. 

 

7) Validate the design. This may just be the most important item of the sevel... Peer reviews, copy known configuration, built it and test it with JetStress (monitor server performance with perfmon counters, 3rd party analyzer tools, etc). Resolve any issues early in rollout.

 


So that's fun, but I'm not done yet. Another thing I've talked with a handful of customers about lately is the disk performance characteristics of various parts of Exchange.

 

As Nicole mentioned in her post, if you consider only logs and database IO, roughly 90% of the IO on the system goes to the databases and only 10% goes to the logs. So right off the top, the database drive performance is a huge consideration in terms of total system performance.

 

The transaction logs are 100% write -- and sequential write at that. Doing nothing but sequential writes to the disk is a comparitively low impact activity. But be mindful of another thing Nicole pointed out: write penalties for various RAID levels. If you have a single disk, that single write IO will take a single IO to the disk. If you have RAID 1+0, it will take two write IOs. And if you put your logs on RAID5, every single write operation will take 4 write IOs at the disk. But, all that said, I've very rarely seen transaction log drives that are dedicated to that task become IO bound and encounter high latency.

 

Database drives are typically going to be somewhere between 66%-75% read IO with the remainder being write. And, of course, this IO is randomly distributed across the disk. Each of the read IOs takes a single IO, but each write (25%-33% of the total IO to the disk) takes 1, 2 (RAID 1+0), or 4 (RAID5) IOs to complete. Nicole ran through a lot of this calculation, so I won't cover it further here.

 

There are a couple of other things that need to be placed on disk somewhere, and the best thing to consider for each is what type of IO profile it most closely matches with. You might be surprised by what you find:

 

SMTP Queues, MTAData, Full-Text Indexes, System TEMP/TMP directories - These tend to be highly randomized reads and writes, so you will find they are most closely aligned with the database drives in terms of IO profile. In fact, placing these on the same disk as the transaction logs -- placing ANYTHING not write-sequential on those disks, actually -- may cause IO latency to increase dramatically on the log disks since they will no longer perform as 100% write-sequential.