The Six Deadly Backup Sins

Get the lowdown on what NOT to do

Microsoft Exchange Server is a complicated product. I can't count the number of times I've heard someone say, "I just inherited our Exchange server, and I need to know what to do." With that in mind, I'm excited to offer this regular column that highlights must-know material for beginning- and intermediate-level Exchange Server administrators. I welcome your comments and suggestions, which you can submit directly to me (at [email protected]) or through Windows 2000 Magazine's Web Forums (http://www.win2000mag.com/support).

A Cautionary Tale
I have a friend who is a talented programmer and UNIX systems manager. He runs a small Exchange server (among other things) for the company that employs him. One of his many strong points is that he knows when he's in over his head, and one day he called me to ask for help. He had shut down the Exchange server, couldn't get the server back up, and didn't have a good backup. The cruel irony of the situation is that he had shut down the server to install a new DAT drive so that he could start making regular backups!

Does this story have a moral? Sure. Backups are crucial, and when you don't have a good set of backups, you're tempting fate. (My friend was lucky—the disk on which the Exchange Server data resided was intact, so I was able to restore the data.) Let's examine the six deadly sins of Exchange Server backup procedures and learn how you can avoid disaster-recovery hell.

Sin #1: Backup Omission
Had my friend installed the tape drive and made a good backup first—before users bulked up the Exchange Server databases—he could have easily rebuilt his machine. In an ideal world, we would all keep up-to-date backups of important data. I bet that all of you reading this column agree that backups are important, but not all of you can truthfully say that you have good backups on hand.

Why don't people routinely back up their Exchange servers? A definitive answer is difficult to give, but one factor seems to crop up more often than you might think: People don't understand how Exchange Server stores data, so they don't know what a successful backup requires.

Exchange Server stores data in three database files: one file for mailboxes (priv.edb), one file for public folder data (pub.edb), and one file for directory data (dir.edb). In addition, Exchange Server associates one set of transaction log files with the directory and another set with the Information Store (IS). For more information about transaction logs, see Tony Redmond, "Exchange Server Transaction Logs," February 1999. Because of the way Exchange Server records incoming transactions, you can often make a full recovery—if you regularly make good backups of the databases and log files. If you lose the databases, you can restore a previous version from backup, then play back the log files to update subsequent incoming transactions. If you lose the log files, you lose some updates but can still restore the majority of your data.

Rule: Perform routine backups of all database and transaction files. Consider how long your organization can go without mail if you don't have adequate backups.

Sin #2: The Wrong Kind of Backup
Shakespeare wrote that a rose by any other name would smell as sweet. Unfortunately, the same isn't true for Exchange Server backups. The type of data you back up and the method you use to back up that data determine the best procedure for your situation. Windows NT and most NT backup software support three useful backup types: full, incremental, and differential. A full backup captures every piece of data, every time. You can use one up-to-date full backup tape to completely restore a server. An incremental backup captures every piece of data that changed since the most recent full backup. You can use a full backup tape (no matter how old) and one up-to-date incremental tape to restore a server. A differential backup captures only the data that changed since the most recent differential backup. You can use a full backup tape and all the following differential tapes to restore a server.

Which mix of backup types you use depends on the size of your Exchange Server IS files, the speed of your backup hardware, and the amount of data you're willing to risk losing. For example, if you perform a full backup every day, the backup process might be time-consuming but you're in danger of losing no more than 1 day's worth of data. If you perform one full backup per week and follow the full backup with daily differential backups during the rest of the week, your daily backup process will take less time. However, the loss of even one differential tape will prevent you from restoring the entire set of data that changed after the full backup (although you might be able to recover some data).

Exchange Server supports two backup methods: online and offline. You can perform an online Exchange Server backup while the Exchange services are running; you can perform an offline Exchange Server backup when the services are stopped. Because the database files are always open while the services run, you must use an Exchange-aware backup program to perform an online backup. The Exchange Server version of Microsoft NT Backup can make an online backup, as can Exchange-specific agents from various software vendors, such as VERITAS Software, Seagate Software, and Legato Systems. Exchange-aware backup utilities also purge transaction log files from the hard disk, a necessary and desirable piece of housecleaning that you must never attempt manually because you can put the database in an inconsistent, and possibly unrecoverable, state. Beware of open-file backup products that purport to back up files that applications are holding open. These products don't produce reliable results with Exchange Server.

The best strategy is to use a combination of online and offline backups. Use an online backup tool to perform your regular backups, and when you stop Exchange services to perform a hardware upgrade or service pack installation, take the time to do a full offline backup first. (Remember that when you restore an offline backup, you must run Isinteg to correct the links between directory and IS objects.)

Rule: When you can afford the time, do daily full backups. When you can't afford the time, use incremental or differential backups, but be careful not to lose or damage the tapes. When calculating how much time you can spend on backups, factor in the amount of time you can afford to be without mail if you don't have good backups on hand. Perform online backups whenever possible and occasionally supplement them with full offline backups before you make major changes to your system.

Sin #3: Circular Logging
Exchange Server creates a new log file for every 5MB of database transactions. When you have a lot of changes to your databases, you get a lot of these log files, which stick around until your next online backup. By default, Exchange Server uses a circular logging mode that overwrites old log files rather than creates new log files. Circular logging conserves disk space but reduces the amount of data you can recover because older transactions aren't available in the logs. Disk space is so inexpensive that you have no good reason to use circular logging (and in Exchange 2000 Server, Microsoft finally turns off circular logging by default).

Rule: Don't use circular logging on servers that store mailboxes or public folders. Period.

Sin #4: Oil and Water Don't Mix
In Sin #1, I mentioned transaction logs and databases. Both file types reside on the hard disk, but the files share no other similarities. Exchange Server sequentially writes log files to disk but makes a lot of small random-access I/O requests for database files. Mixing random and sequential I/O on one hard disk hurts performance. Worse still, putting the transaction logs and databases on one disk decreases your chance of a successful recovery because if that disk fails, you can lose both the logs and the databases.

Rule: When possible, store your transaction logs and databases on separate physical disks. Storing these files separately is probably the easiest and most cost-effective way to quickly boost server performance and recoverability. And remember, you still need to make regular backups.

Sin #5: Information Store Gluttony
Don't let your IS files get too big before a backup, or you'll end up moaning "I can't believe I ate the whole thing" as you wait for a backup or restore to complete. For best performance, you need to run backups when users aren't actively using the server, so you'll probably back up your Exchange Server during off-hours. If you can't complete a backup during that time, you have a problem. And restoring Exchange Server data from tape takes approximately twice as much time as backing up the data (not counting the time that you might need to restore the underlying OS and machine configuration). If your backup takes 8 hours to complete, then you'll need at least 16 hours to restore the backup. Can your organization tolerate that much of a wait? You need to find out beforehand. Use the backup hardware and software you have in place to perform an online backup so that you can calculate how much time you need to perform an offline backup or restore.

Rule: When your backups take too much time, your restores will, too. Get faster backup hardware, change your backup strategy, or split user mailboxes and public folders between servers so that your data is distributed. (Be sure to compress the IS after you finish moving these items.)

Sin #6: If a Tape Fails and No One Hears...
How do you know that your backups are working properly? Do you regularly check the system event log? Do you regularly review your backup software's log files to look for failures? When your logs don't indicate problems, are you confident that the logs are showing the truth? I once visited a company that had one production server with one DAT drive—the only DAT drive on site. When the server failed, the company found that the DAT drive had a hardware failure. The company couldn't restore the server until a replacement DAT drive arrived. Oops.

Dirty, misaligned, or improperly configured tape drives are frequent offenders. The only way you can be absolutely sure that your backup processes and procedures are working correctly is to restore your backup tapes onto a recovery server. This precaution might seem unnecessary, but the first time you catch an error, you'll be delighted that you performed this task.

Rule: Occasionally perform a restore to test your backups. You'll verify your backup procedures and gain valuable practice in the arcane art of Exchange Server disaster recovery.

Go Forth and Sin No More
Now that you know what not to do, you can fix the shortcomings in your backup procedures or server configuration before those deficiencies turn into headaches. The rules I've suggested are easy to implement and can save you a lot of trouble down the road.

Corrections to this Article: