Enabling Message Journaling on Exchange Server

In Exchange Server 5.5 Service Pack 1 (SP1), Microsoft added a new feature—journaling—that captures copies of user's messages within the Exchange system. Journaling lets an administrator capture all messages to another recipient (i.e., mailbox, custom recipient, or public folder) as soon as anyone submits or receives the message.

Although most private companies want to delete messages after a week or 2 for various reasons (e.g., to prevent litigants from introducing messages as evidence in criminal or civil lawsuits), some companies and government agencies require retention of every message sent through an organization. For example, the Securities and Exchange Commission (SEC) requires retention of messages that relate to a broker-dealer's business, and the State of Florida requires its agencies to store all email messages for public record retrieval under the Florida Sunshine Law.

Journaling is different from archiving. Journaling saves messages to a recipient, whereas archiving moves those messages to long-term storage where you can search for and retrieve them. Here, we describe how message journaling works and how to configure it. In a future issue, we'll review options (e.g., Microsoft Exchange Server Archiving Agent, third-party utilities) for archiving the messages you've captured in your journal.

Message Journaling Internals
For Exchange to capture all messages sent through an Exchange server, every message, regardless of destination, must travel through the Message Transfer Agent. The MTA looks at the message-repudiation field in the P1 header of all email messages to determine whether a message working its way though the system is a candidate for journaling. (The MTA uses the message-repudiation field primarily to detect message looping.) Every MTA involved in delivering a message puts a stamp on that message to signify whether Exchange journaled it or not. Then, the next receiving MTA determines whether it needs to journal the message based on the Per Site Registry setting.

The requirement that the MTA be involved in journaling creates a challenge in two scenarios. First, if both the originator and destination recipient are on the same server, messages typically bypass the MTA, and the Information Store (IS) handles the message exclusively. Second, if the Internet Mail Service (IMS) receives an inbound message and the destination recipient is on that server, the IMS usually hands off the message directly to the IS without the MTA's involvement. In both cases, you must modify the process so that the MTA can journal the message.

Because Microsoft didn't include a GUI for message journaling, you must edit the Windows NT Registry to add the appropriate values to the MTA, IMS, and IS settings. To disable local delivery and force the MTA to handle the message, add the REG_DWORD data type value No Local Delivery to the HKEY_LOCAL_MACHINESYSTEM CurrentControlSetServices MSExchangeISParametersSystem key. Enter 1 in the Data Value field. Screen 1 shows the No Local Delivery entry.

To force all Internet messages to pass through the MTA, add the REG_DWORD data type value ReRouteViaStore to the HKEY_LOCAL_MACHINESYSTEM CurrentControlSetServicesMSExchangeIMCParameters key. Set the value to 1. (Make sure you do not have spaces between the words ReRouteViaStore.) You need to add this string value to only those servers that have the IMS installed locally.

Message journaling doesn't capture most system-based messages such as directory replication, public folder replication, and link monitor messages. Journaling also ignores some user-initiated messages. For example, journaling doesn't capture messages posted to a public folder (and therefore not sent to the public folder's email address) because the store process inserts those messages directly into the public IS.

Message journaling requires additional system resources; it increases system load about 10 percent to 15 percent, depending on hardware configuration and message load. For example, journaling increases the MTA's workload because journaling routes all messages through the MTA. In addition, the IMS and the IS service use additional resources to process local messages. If you place the journal recipient on a remote server, all client-based messages must use network bandwidth to send a copy of that message to this server. Because Exchange tags journaled messages as low-priority system messages, Exchange transfers other user messages first. Use the Windows NT Performance Monitor to set benchmarks before you enable journaling, and check the benchmarks after you begin journaling to ensure your system can handle this additional load. If you find stress in your system, you can upgrade your hardware accordingly (e.g., by adding memory).

Planning for Message Journaling
Before you implement message journaling, you need to determine which users' or servers' messages you need to capture (journaling scope) and the type of recipient (mailbox, public folder, or custom recipient) you want to store the journaled messages. You also need to plan how you'll remove the messages from Exchange for long-term storage (archiving) and how you'll search for and retrieve messages from the archive (retrieval). However, this article doesn't discuss archiving and retrieval.

Scope of message journaling. Do you want journaling to capture individual copies of messages at the Exchange organization level, the site level, or the server level? By default, message journaling occurs at the organization level. At this level, Exchange keeps one copy of the message, no matter how many sites and servers it passes through. If you configure message journaling at the site level, the MTA captures one message for each site the message goes to. You can also have each server retain a copy of every message, but this retention is usually useful only if you want to capture all mail on a bridgehead or connector server. Although you can set any level of journaling at each Exchange server, using the same scope setting on each server is best for consistency and reliability.

Choosing a journal recipient. The journal recipient is an Exchange object you choose to have the local MTA deliver the journaled message to. The type of recipient you choose is one of your most important decisions because it affects your database size. You can choose a mailbox, public folder, or custom recipient. Usually, the recipient is a mailbox or a public folder because making the journal recipient a custom recipient means you're sending the messages to a foreign system outside of Exchange. A custom recipient might be an appropriate recipient if you want to use SMTP to send all the mail to another server at a different location for offsite storage or disaster recovery.

Using a mailbox as the journal recipient lets you preserve single-instance message storage for any messages sent locally to that server. Single-instance storage lets multiple users reference one copy of a message and preserves precious disk space. The single-instance store is beneficial primarily if you have one Exchange server and don't want to increase your storage requirements. When you have multiple servers, other factors such as the journaling scope and journal recipients become more important than the single-instance store.

Using a public folder as the recipient defeats single-instance storage because Exchange must store a copy of the message in the public store, and the public store is a separate database from the private store. However, using a public folder has advantages. You can control public folder replication and thus determine the specific times of day the journaled messages create traffic on your network. You can also automatically delete older messages when you no longer need them. This feature is useful if you want to retain journaled messages for a short time (e.g., 30 days). In summary, public folders have a more diverse set of configuration options that enhance your control over the journaling process.

If you use a public folder as the journal recipient, change the public folder's permissions so that only authorized personnel can view the contents. By default, all recipients in the Exchange organization have the Author role, which lets them read the public folder's contents, including everyone's email messages. Therefore, you need to change the default permission to Contributor to ensure that the MTA process can journal messages but users can't see the folder or messages within the folder. To change the public folder's permissions, go to the Journaling Properties page, which Screen 2 shows. In addition, you need to disable any IS storage warnings or limits on the journal folder; you can override this global setting by setting a higher threshold on the public folder's Limits property page.

Whether you use a mailbox or a public folder, hide the recipient from the Global Address List (GAL) for security. Otherwise, users might accidentally send mail to that recipient or post messages in a public folder, adding duplicate messages to your journal. Exchange hides public folders from the GAL by default. To hide a mailbox from the GAL, go to the Advanced tab of the mailbox's property sheet and select the Hide from Address Book check box.

Choosing a Journaling Architecture
The way you set up your journaling structure and choose the journal recipients depends on the topology of your Exchange environment. You have several options.

Option 1. Configure all servers to journal their messages to one centralized location, essentially doubling the number of messages transferred over the LAN or WAN. This option creates a central location for search and retrieval. For example, you can define one mailbox as the journal recipient.

Option 2. Configure all servers to journal messages locally, then archive those messages locally. This approach has the least impact on network bandwidth, but it also creates a distributed archive that makes locating a message more difficult.

Option 3. Configure all across-the-WAN servers to journal messages to a locally created public folder that replicates centrally to a master server after business hours. You can archive the contents of these public folders at this central server. This approach lets you control the impact of the journaled messages and bandwidth utilization, and it creates a central location for search and retrieval.

Option 1 is best suited for a small number (i.e., fewer than 10) of servers connected over high-speed networks because each server requires additional network bandwidth. In a multiserver organization, this approach might be unwieldy, because you have no control over the messages to this recipient. The MTA copies and transfers the messages in realtime, and you can't hold the messages in a queue for transfer during nonbusiness hours.

Before you choose this option, determine the bandwidth requirements that Exchange is placing on your network, and triple those requirements. Measure the increased bandwidth requirements against your service level agreement (SLA) message delivery times to see whether you can still meet them under peak demand. If the increased bandwidth requirements will slow your delivery times to unacceptable levels, option 3 is your best choice.

Option 2 is a good choice in a highly distributed environment, especially in an installation that has a large number of Exchange servers that communicate across WAN links. Because each server journals the messages locally, the increased demands are on local storage, not on the network. This option requires that you regularly archive messages at each server to remove the messages from the journal recipient and run another process to get the messages off the server for long-term retention. Many tape backup software packages have a file-grooming feature, which lets you move the archived messages to tape to free up disk space.

Option 3 is a hybrid of the other options and leverages the public folder replication process to control how the message traffic affects network bandwidth. This option is a good choice if a centralized archive is crucial to your business needs but your network won't support option 1. This option controls bandwidth needs and schedules replication at night for the least impact, but it concentrates replication in a shorter time frame. This option doesn't work well when the volume of information that users create at the remote servers is too large to move to a central location; option 2 might be a better choice in that situation.

Configuring Message Journaling
After you decide on your journaling architecture, you need to configure your system. The best way to show you how to configure a system is with an example. Suppose your organization consists of four servers (Cincinnati, Columbus, Cleveland, and HQ) in one site. You decide to use option 3 because you need to control traffic and need to use an architecture that has the least impact on the WAN during business hours.

One possible configuration is to have the regional servers (Cincinnati, Columbus, and Cleveland) replicate their public folders to a master centrally based server (HQ) at night when mail traffic is light. With this configuration, you can schedule replication just after the backup routine. The master server can then use one of several archiving programs to archive all the public folders (including its local public folder) to disk. (We'll discuss these programs in a future article.) After you use the archive program to remove messages from the public folder, the next scheduled public folder replication cycle will delete messages from the folders on the remote servers.

You decide to use this configuration with public folders as the type of journaling recipient. Now, you must obtain the distinguished names (DNs) for each server's public folder. To view the DN,

The DNs are different for each public folder because they depend on the organization and site name you use. Table 1 shows their values in this example.

Now you're ready to enable message journaling. First, install Exchange Server 5.5 SP1 or later on any servers in your Exchange organization that you want to participate in journaling. Next, edit the NT Registry to configure message journaling, according to the following instructions:

After you've reviewed your edits for accuracy, close the Registry. Message journaling will take effect immediately.

The Results
Screen 4 shows what the journal recipient has captured. As you can see, you captured messages from a Microsoft Mail (MS Mail) client (Administrator), a Lotus cc:Mail client (ccChris), and a POP3 mail client (Eudora). In addition, local delivery is apparently passing through the MTA because the Student1 and HQ-Journaling recipients reside on the same home server.

The recipient will display Read receipts, but by design, journaling doesn't capture delivery receipts. Also note that the recipient recorded some system messages: a Mail failure message and a notification that a message is queued up on the IMS waiting for a retry.

What Message Journaling Can't Do
Message journaling has some limitations. For instance, it can capture encrypted messages, but you still need a private key to open them. Journaling won't capture POP3/Internet Message Access Protocol 4 (IMAP4) clients that use an Exchange server for their mailbox location but use a non-Exchange SMTP server. You need to point these clients toward the Exchange IMS for their outbound SMTP server so that journaling can take place. Also, MS Mail and cc:Mail client messages must pass through Exchange journaling to work; therefore, these messages must travel across the cc:Mail or MS Mail connector for Exchange to capture them. If these messages pass through only cc:Mail or MS Mail post offices, journaling won't occur.

A Useful Process
Not everyone needs message journaling. Unless you have a government obligation to save messages or a corporate policy that dictates archiving mail, you might not need this function at all. However, if you must keep all your messages, message journaling provides a quick way to capture them.

Journaling is only half of a total archival solution. You also need some process to archive, search, and retrieve those messages. With the added functionality of the various archiving solutions, which we'll discuss in a future article, you'll have a quick way to consolidate those journaled messages into an indexed storage location for easy retrieval.