Storing Exchange Server Messages

Some systems administrators use Exchange Server personal folders to store mail messages. Often, these administrators want to restrict the size of each user's mailbox because their servers don't have much disk space available for the Exchange Information Store (IS). Although personal folders (i.e., personal message stores, or .pst files, that reside on a file system that an Outlook client can access) are useful in some instances, relying on them might not be the best strategy. Using server-based storage (i.e., the IS that resides on the server) is more advantageous in most circumstances.

To back up my preference, I'll explain why server-based storage is superior in five areas: single instance storage, message delivery, mail security, user access, and mailbox recovery. Then I'll point out some good uses for personal folders.

Single Instance Storage
Single instance storage is Exchange Server's system for storing messages. In this system, Exchange Server manages one copy of a message's content and attachments and uses pointers to let multiple users share the message. (For more information, see "Inside the Exchange Information Store," Windows NT Magazine, April 1998, or enter InstantDoc ID 3153 at http://www.winnetmag.com.)

If you use personal folders, Exchange Server can't use single instance storage. Instead, Exchange Server must create separate copies of messages for each user who wants mail delivered to a .pst file. Worse yet, the amount of data doubles because a personal folder holds two copies of each message (one copy in Rich Text Format—RTF—and the second in plain text), to accommodate users who use clients that can't understand RTF (e.g., the old DOS client for Exchange) to read mail. You end up with duplicated messages and more data in storage.

Slow Message Delivery
Using personal folders slows down message delivery. Clients must connect to a server to download mail before you can process mail. This connection isn't a serious problem if you always access the server over a LAN connection. However, downloading messages over a dial-in link can be very slow, especially when many messages are in the queue for downloading. You have no control over what the client downloads, so if the first message that's waiting has a 10MB attachment, your dial-in connection can last for a long time.

Mail Security
Personal folders are not secure. A .pst file is a friendly file format that any Exchange or Outlook Messaging API (MAPI) client can open, so anyone who browses your disk (or personal share) and finds a .pst file can open it. Users can protect their personal folders with a password, but rescuing users who forget their password is a hassle because Microsoft doesn't let its support technicians help break open encrypted user files. Instead, you'll need to use a utility that opens encrypted .pst files. You can find such utilities on the Internet, which means that hackers can download the same utility to open users' .pst files.

Server-based storage also has a security weak spot. Systems administrators can read any mail users store on the server. However, you can encrypt confidential messages using Exchange Server's standard Advanced Security features or other secure add-ons for clients, such as MailSecure from Baltimore Technologies (http://www.baltimore.ie). Or you can do what most sensible organizations do: Threaten to fire systems administrators who poke around in other people's inboxes. Every time a Windows NT account opens a mailbox and that account is not the primary account registered for that mailbox, Exchange Server logs an event in the application event log. Examine logs routinely to ensure that administrators aren't browsing inappropriately. Note that because many store-based antivirus agents log on to everyone's mailboxes to check incoming messages, you might see more of these events than you might expect.

Although hackers can crack encrypted messages in personal folders, hackers can't open encrypted messages secured with Advanced Security. You need personal keys to decrypt messages, and you don't have access to those keys unless users hand them over (or write them down on a piece of paper with all their other passwords).

User Access
Administrators who use personal folders for primary message storage typically place personal folders on personal network shares to let users access their messages from multiple PCs. However, only MAPI clients can open .pst files, so if you use Post Office Protocol 3 (POP3) or Internet Message Access Protocol 4 (IMAP4) clients or a Web browser, you won't be able to access your messages.

The inability to use a Web browser is an important consideration. Many times, I've found the network connection to my server to be slow or a suitable client unavailable to let me check mail. Outlook Web Access (OWA—the server-side application that lets any Web browser that supports frames and JavaScript open an Exchange Server mailbox) is a great way to check mail when you need to use extended links or when a MAPI client isn't available. PCs now typically come with a Web browser, so theoretically you can fire up Internet Explorer (IE) or Netscape Communicator on someone else's PC and read your mail. But if you have your messages delivered to a personal folder, you can't access your messages.

Mailbox Recovery
Recovering individual mailboxes or items is easier with personal folders than with server-based storage. To restore a specific file from a backup tape, you need to know only the name of the .pst file where Exchange Server delivers messages. You can back up .pst files with any backup software. Always back up personal folders when clients are not likely to be active, because if a client is using a .pst file, the backup software might not be able to access the file and write a good copy to tape. If someone leaves the office with a PC still turned on, the client automatically disconnects after 30 minutes to let backups proceed.

You can recover server-based mailboxes easily only if you have taken bricked backups with software that supports this feature, such as Computer Associates' ARCserve 6.5. A bricked backup is different from a regular backup because it writes contextual information to tape alongside raw data. In the case of Exchange Server, the context represents the folder structure inside a mailbox; the restore utility can read this structure, if you need to extract one mailbox from the backup tape. However, bricked backups take perhaps four times longer than regular, streamed backups.

On the surface, personal folders' easier restores look like an important advantage, but when you consider the following facts, the advantage isn't as obvious. First, if people are using personal stores, you might need to do more restores, because users might delete a .pst file without thinking. However, users can't usually delete the databases that make up the IS.

Second, the deleted items recovery feature in Exchange Server 5.5 has eliminated almost all the problems arising from users deleting messages or other items by mistake. Third, bricked backup software for the IS is now available.

Finally, recovery is impossible without good backup discipline. Without established procedures for performing backups, verifying their contents, and keeping backup tapes in a secure location, any data is at risk. With server-based storage, you have one entity to back up rather than multiple files spread over potentially large numbers of disk volumes. People get paranoid (and rightly so) about the possibility of losing data as a result of a corrupt IS. However, in most situations, systems lose data because either the systems administrators didn't make backups regularly or the hardware was faulty. These causes are common with both personal folder and server-based storage. The lesson here is to use the highest quality hardware (i.e., disks and controllers) for data storage and to take backups every night.

Some Good Uses for Personal Folders
Even a server bigot like me can find some value in personal folders. I like using personal folders to store information that I don't want to keep on the server. For example, I commonly download large attachments into a .pst file to free space in my mailbox quota. The autoarchive feature in Outlook is also a useful way to move obsolete messages offline on a regular basis. Screen 1 shows the feature in action. In this case, I used the Tools Archive option to tell Outlook to move all my inbox messages older than April 1, 1998, to a personal folder. Outlook 98 has made significant performance improvements in this area, and now autoarchive is a usable feature.

Outlook 98 moves the items in the background, so the process doesn't affect your regular work. You can also archive folders automatically by instructing Outlook to check each folder's autoarchive setting regularly, as Screen 2 shows. The Tools, Options, Other option controls how Outlook performs archiving, although each folder can have an autoarchive setting. Setting autoarchiving per folder is useful, because you might want to keep some items in server folders longer than others. For example, sent items aren't usually useful after 60 days, but you might want to keep documents relating to a project for much longer. Unfortunately, personal folders don't support read-only devices. You might want to move personal folders to jukeboxes or similar hardware for long-term archiving, but MAPI clients insist on opening .pst files with write access.

Personal folders are the best way to share information between users when no network connection is available. You can create a .pst file on a disk, drag the items you want to share, and pass on the disk. Note, however, that even if a .pst file is on a shared file service, you can't share a personal folder between two clients because .pst files don't support concurrent access. Finally, Exchange Server's Move Mailbox option can move mailboxes only between servers in a site. If you want to move a mailbox to another site, you must move all the messages out of the mailbox to a personal folder and import the messages back to the server (if you want) when you create a new mailbox.

Use the Server
Personal folders aren't as useful as they were in the past. As a result, consider using server-based storage. If your server doesn't have enough disk space for large mailbox quotas for users, you need to buy new disks. Disks are continually getting bigger, faster, and cheaper.

Exchange Server 5.5, Enterprise Edition eliminates the old 16GB restriction on the size of an Exchange Server database, so you can add as much disk space as you want and keep everyone's mail on the server. However, don't let your enthusiasm run away with you. Look at how utilities such as the Mailbox Cleanup Agent (available in the Exchange Server Resource Kit) can help keep mailboxes trim. Also, pay attention to backup times, and make sure that you have the right combination of backup software and hardware to ensure that backup operations complete quickly, every night.