10 Tips to Keep Your Microsoft Exchange Server Humming

Executive Summary:

Because it's been around for well over a decade, Exchange Server is a mature product, and Exchange Server 2007 seems to be more stable and reliable than its predecessors. Even so, every version of Exchange requires occasional, regular maintenance. So here are some tips for keeping your Exchange servers running as smoothly as possible.

Run ExBPA Periodically
One of the best pieces of advice I can give you is that you should periodically run the Microsoft Exchange Server Best Practices Analyzer (ExBPA). Running it every month or two should be sufficient for most organizations. In case you aren't familiar with this valuable tool, ExBPA analyzes your Exchange organization's configuration and compares it to Microsoft's recommendations. Administrators typically run ExBPA after deploying a new Exchange organization, then use the tool to help them fine-tune the organization. After everything is correctly configured and optimized, though, people tend to forget all about ExBPA.

There are at least two good reasons to continue running ExBPA periodically. First, you should run the ExBPA any time you add a new server to your Exchange organization. Not only do you need to verify that the new server is configured appropriately, but also adding a server can have a negative effect on the performance of your Exchange organization as a whole. Running ExBPA after deploying a new server can help you avoid such growing pains.

The second reason you should run ExBPA on a periodic basis is because Microsoft is notorious for changing their recommendations. I've lost count of the number of times over the years that I've written about a technique in an article, only to receive an email message a month or two later from someone telling me that Microsoft has changed their recommendations.

Don't get me wrong: These changes aren't a bad thing. If Microsoft discovers a more secure or more efficient way of configuring Exchange, I want to know about it. Fortunately, every time ExBPA runs, it checks a centralized database that Microsoft updates with its latest configuration recommendations. Therefore, if you run ExBPA periodically, you might find ways to modify your Exchange organization to make it more secure or efficient.

Configure Database Maintenance Schedules
As messages are sent, received, moved, and deleted, the Information Store is bombarded with transactions. Over time, all this activity can leave the database in a sad state of fragmentation. Fortunately, Microsoft knows this and has designed Exchange Server to run an automated maintenance cycle each night.

I won't go into all the details of how the maintenance process works, but here are the eleven maintenance tasks that Exchange attempts to perform in each nightly cycle:

The first ten tasks on the list are treated with equal importance, but the eleventh task, defragmenting the Store, is considered to be far more important than the other tasks.

The automated maintenance occurs within a specified time slot each night. When the maintenance process begins, Exchange starts with the first task, then works its way down through the list. If all the tasks aren't completed within the allotted time period, Exchange notes which tasks have finished and which tasks have yet to be completed. The next night, the maintenance cycle begins with the first incomplete task from the night before.

The big exception to the rule is online defragmentation. As long as at least one maintenance task has completed, Exchange dedicates the last fifteen minutes of the maintenance cycle to defragmenting the database. This happens even if some tasks didn't have time to complete. If defragmentation hasn't completed by the end of the specified time slot for automated maintenance, Exchange extends the maintenance cycle by up to an hour to ensure that the defragmentation has time to complete.

In both Exchange 2007 and Exchange Server 2003, you can check the event logs for event ID 700 (online defragmentation started) and event ID 703 (online defragmentation completed). Exchange 2007 offers several Performance Monitor counters to monitor the online defragmentation process, and two additional counters have been added in Exchange 2007 SP1. You can read about the individual counters in the Microsoft article "How to Monitor Online Defragmentation" (http://technet.microsoft.com/en-us/library/bb691410.aspx).

Because this maintenance process is automated, you might think there's nothing you need to do. However, the default time period used by Exchange for nightly maintenance might not be the best choice for your environment. Keep in mind that the maintenance process is resource intensive. It's therefore important to schedule the maintenance for a time when the server has a minimal workload, and at a time when it won't interfere with the nightly backup. If your Exchange server has multiple Stores, you might want to schedule the maintenance to occur at a different time for each one. For a list of articles on defragmentation, see the Learning Path at the top right of this page.

To adjust the scheduled maintenance period in Exchange 2007, open Exchange Management Console and navigate through the console tree to Server ConfigurationMailbox. Automated maintenance is scheduled for each database individually, so right-click the database you want to adjust, then select Properties from the resulting shortcut menu to open the Store's Properties dialog box. As you can see in Figure 1, Exchange is configured by default to run the automated maintenance cycle from 1:00 a.m. to 5:00 a.m. Click Customize if you want to change the maintenance schedule for the database.

If you're using Exchange 2003 or Exchange 2000 Server, you would use Exchange System Manager to adjust the scheduled maintenance period. Navigate through the console tree to your Store, right-click it, and choose Properties from the shortcut menu. Go to the Database tab on the Store's properties sheet, and click the Customize button.

Perform Integrity Checks
Exchange performs the automated maintenance tasks on its databases every night. You should still perform a manual integrity check on a quarterly basis. Manual checks let you see if there are any problems with the databases and take corrective action if necessary.

Before performing an integrity check, make sure you have a full backup of the database. In rare situations, performing manual database maintenance can cause database corruption. You also need to make sure you have adequate disk space. If you have to perform any type of repair on the database, you'll need enough free space on the volume for a full copy of the database, plus another 10 to 20 percent for overhead.

To perform a server-level integrity check, you first need to dismount the Store. In Exchange 2007, open Exchange Management Console and navigate through the console tree to Server ConfigurationMailbox. Next, right-click the database you want to check and select Dismount Database from the shortcut menu. To dismount a database in Exchange 2003, open Exchange System Manager, navigate through the console tree to your store, right-click it, and choose Dismount Store from the shortcut menu.

After the database is dismounted, you can use Isinteg to check for errors in the database. Open a command prompt window, navigate to the Program FilesMicrosoftExchange ServerBin folder, and enter the following command:

isinteg -s  -test allfoldertests

When you run this command, you'll receive a list of the databases on the server, as Figure 2 shows. Next, enter the number from the list for the database you want to test. Isinteg prompts you for confirmation; press Y to start the tests. If any errors are reported, Isinteg tells you what corrective action to take, and you should perform such actions right away.

If the server-level integrity check with Isinteg doesn't return any errors, you should perform a database-level integrity check by using Eseutil. To do so, enter the following command:

eseutil /G ""

In the above command, you would replace with the actual database path (in quotes). For example, the command might be

eseutil /G "Q:program Files
     MicrosoftExchange Server
     Mailbox First Storage GroupDatabase.edb"

Isinteg and Eseutil work the same in Exchange 2007 and Exchange 2003.

Check Your Database for Free Space
As already mentioned, Exchange defragments its databases as a part of the nightly maintenance cycle. However, an online defragmentation doesn't actually shrink the size of the database. Instead, empty database pages, known as free space, are simply grouped together so they can be efficiently reused.

Usually this technique doesn't present much of a problem, but there are circumstances when you might need to shrink a database. For example, if you moved some mailboxes to a different store as a way of freeing up disk space, you wouldn't accomplish your goal unless you ran an offline defragmentation afterward.

Even if you aren't trying to reduce the amount of space consumed by your databases, it's a good idea to perform a quarterly check to make sure that the databases don't contain excessive amounts of free space. Generally, free space is considered to be excessive if it occupies more than 15 percent of the total database. The easiest way to find out how much free space is in a database is to search your server's application log for event ID 1221. As you can see in Figure 3, the Event Properties dialog box tells you how many megabytes of free space are in the database. Use this number along with the database's total size to figure out the percentage of free space.

If you need to remove free space from a database, you can do so with the Eseutil command. You'll have to dismount the database first, and be sure to follow the earlier words of caution about having a full backup and enough disk space for a backup copy. You would enter the command

eseutil /D ""

where is the actual database path.

Verify Your Backups
Verifying your backups isn’t really an Exchange maintenance task, but it's so important that I just couldn’t leave it out. Like any other maintenance task, testing your backups is something that needs to be done regularly.

Back in the late 1990s, I had a client who had a rather large Exchange Server 5.5 deployment. As you would expect of any enterprise-class organization, this organization backed up its Exchange servers religiously—and spent a lot of money on the best backup software and hardware. Eventually, one of the Exchange servers crashed because of database corruption. When the client's systems administrators attempted to restore the backup, they discovered that the corruption had actually begun months earlier and had grown progressively worse over time. Therefore, there wasn't a good backup available, and all the data on that server was lost.

If the administrators had taken the time to test their backups (and to perform regular database integrity checks), they would have realized earlier that there was a database problem and could have taken steps to correct it before it became catastrophic. Granted, Exchange 5.5 was much more prone to database corruption than Exchange 2003 and Exchange 2007. Even so, the lesson is just as relevant today as it was then.

I work with a client now that uses NTBackup to back up its Exchange server and file server. This organization does periodically test its backups. However, we've recently discovered that NTBackup has a bug: .bkf files over a certain size are basically rendered invalid. The bug crops up when the backup file size grows to somewhere between 150GB and 250GB. Something in the way NTBackup creates the file causes the problem; it's not a file-size limit. NTBackup doesn't report any problems when the backup is created; and had my client not been testing its backups, it might never have known about this problem until it was too late. There isn't a Microsoft Knowledge Base article on the subject yet.

My advice is to test your backups monthly. You can restore your backups to lab servers to test them.

Verify Automatic Maintenance Tasks
Although Exchange's regular eleven maintenance tasks occur automatically, it's still a good idea to check once a week or so to make sure all the tasks are finishing regularly. Unfortunately, there isn't a single place you can look to verify that the maintenance cycle has completed successfully. There are, however, some telltale application log entries you can use to figure out the state of the automated maintenance cycle. Here's a list of the event-log entries of interest:

Keep Track of Database Growth
One easy thing you can do to keep your Exchange server in good shape is to document database growth on a monthly basis; you might easily incorporate this task with verifying your backups. You should document the size of the most recent backup as well as how long that backup took to complete. This data will help you anticipate your server's future storage requirements. For example, monitoring growth trends might help you see ahead of time if your backups are going to take longer than your allotted backup window, or if you're in danger of exceeding the backup media's capacity.

Check the Queue for Stuck Messages
Occasionally, messages become stuck in Exchange’s queues. Therefore, it's a good idea to check the queues every few days for any stuck messages that need to be removed.

You can view the message queues in Exchange 2007 by opening Exchange Management Console and selecting the Queue Viewer from the Toolbox, as Figure 4 shows. In Exchange 2003, you use Exchange System Manager to view message queues. Navigate through the console tree to Administrative Groupsyour administrative groupServersyour serverQueues. Select the Queues container, and the Queue Viewer will be displayed.

If you find a stuck message, you can use message tracking to troubleshoot the problem, or you can just delete the message. Exchange gives you the option of sending a nondelivery report (NDR) for deleted messages.

Test Your UPSs
Another maintenance item that isn't strictly Exchange-related is testing your UPSs, which I suggest doing each month. Typically, if power fails, the UPS should instantly engage and keep your servers running. A good UPS is worth its weight in gold, but the problem is that many people don't test them regularly. A UPS doesn't last forever; over time, its battery capacity diminishes.

A UPS should be configured to begin shutting down a server gracefully if its batteries begin to run low. Over the years, I've seen several situations in which a UPS initiated the shutdown of an Exchange Server but the battery failed before the shutdown was complete. I've seen databases severely corrupted because of a power failure occurring during a database dismount. As I mentioned earlier, Exchange 2007 is much more resilient to this type of corruption than older versions of Exchange, but why take a chance? Rather than wondering what your aging UPS will do during the next blackout, you should test it once a month.

To test your UPS, simply dismount the Exchange databases so that no data corruption will occur accidentally, then pull the plug and see what happens. Most organizations don't look too favorably on taking mail servers down. Therefore, try to schedule UPS tests to coincide with scheduled reboots or other scheduled maintenance.

Invest in Ops Manager
If you're reading this article, you're probably already familiar with Microsoft Operations Manager (MOM) or its latest update, System Center Operations Manager 2007. I recommend investing in one of these solutions as perhaps the most crucial step in keeping your Exchange organization running efficiently.

Ops Manager monitors an Exchange server's various performance counters and event-log entries, and helps you make sense of the data. It tracks resource utilization and server response time, and alerts you to situations that could eventually cause problems, letting you take proactive corrective action. You can learn more about Ops Manager on Microsoft's Web site at http://www.microsoft.com/mom/default.mspx. Many third-party monitoring solutions are also available; you can find out about some of the best of them in the Windows IT Pro article "Exchange Server Monitoring Tools" (http://www.windowsitpro.com/Article/ArticleID/97151/97151.html).

Be Proactive!
Exchange 2007 is designed to be reliable without excessive administrator-initiated maintenance. Even so, it's a good idea to be proactive and perform various maintenance tasks regularly to keep your server in top shape. When you're using an older version of Exchange, these tasks become even more important. But, if you establish regular maintenance procedures based on these 10 tips, you should be able to avoid common problems and keep your servers humming.