Exchange 2003 Advanced Journaling

In "An Exchange 2003 Journaling Primer" (April 2005, InstantDoc ID 45348), I described the fundamental differences between journaling, archiving, and compliance, and I talked about the most basic form of journaling: message journaling. However, message journaling might not provide the functionality you need for a comprehensive compliance system. For example, what if you need to support distribution list (DL) expansion and deal with message reports?

Exchange Server 2003 provides another form of journaling: envelope journaling. Envelope journaling is a more sophisticated form of journaling and is generally the most common form implemented by organizations that are serious about journaling. Let's take a close look at envelope journaling—how to enable it in both Exchange 2003 and Exchange 2000 Server and how to make it work in a multiserver environment.

Envelope-Journaling Benefits
Message journaling simply captures a copy of any messages sent from or received by a mailbox in a database on which journaling has been enabled, then forwards those captured messages to the journal mailbox. Message journaling looks only at the P2 message headers and uses those headers to determine the recipients and originators of the message. (See "An Exchange 2003 Journaling Primer" for information about both P1 and P2 headers.)

However, envelope journaling inspects the P1 message headers on the message envelope. The envelope contains the definitive routing information for the message. The headers contain the actual SMTP addresses that the Exchange transport system uses to route and ultimately deliver the message. The P1 headers contain any recipients that may be blind carbon copy (Bcc) recipients. In addition, P1 headers explicitly identify the SMTP addresses for recipients who are members of any DLs to which the message might have been sent. As such, the P1 headers authoritatively define to whom messages have been transmitted. For more information about P1 and P2 message headers, see the Internet Engineering Task Force (IETF) Request for Comments (RFC) 821 and RFC 822, respectively.

Configuring Envelope Journaling
Exchange 2003 supports envelope journaling only if all Exchange 2003 servers in your organization are running Exchange 2003 Service Pack 1 (SP1). There's no check for SP1 servers during configuration, but you'll experience inconsistent behavior with envelope journaling if non-SP1 servers exist in your organization. Enabling envelope journaling in your organization is a two-step process. First, you must enable message journaling for any Exchange databases on which you want to journal messages that are either sent or received by mailboxes homed in that database. (For details about this process, see "An Exchange 2003 Journaling Primer.")

Second, you must enable envelope journaling in your organization. To do so, go to Microsoft's Downloads for Exchange Server 2003 page (http://www.microsoft.com/exchange/downloads/2003/default.mspx) and download the E-Mail Journaling Advanced Configuration tool—aka Exejcfg—to a convenient directory on any Exchange 2003 server in your organization. This tool is also available in the Exchange 2003 SP1 kit, in the i386rtw folder. The download is a self-expanding executable file that contains a license file, a user guide, and the exejcfg.exe file. From a DOS command window, you can enable envelope journaling by typing

exejcfg.exe -e

where the -e qualifier indicates that envelope journaling should be enabled. You can disable envelope journaling by using the same command with a -d qualifier. Using the -l qualifier displays the current status of envelope journaling. Figure 1 shows the result of enabling envelope journaling in my test environment.

The Exejcfg utility merely sets an attribute value on the Exchange organization in Active Directory (AD). The Exchange organization's heuristics attribute typically has the value 0, but when envelope journaling is enabled, this attribute has the value 512, as you can determine by using ADSI Edit as Figure 2 shows. (You don't actually need to use the Exejcfg tool for this task. You could simply use a utility such as ADSI Edit or LDP, which can modify AD attributes, to enable envelope journaling.) Because you're applying an organizational setting, envelope journaling is enabled for the entire organization. Thus, you can't enable a combination of message journaling and envelope journaling. However, in general, envelope journaling is considered superior to message journaling.

If you've been using message journaling and have used the registry modification outlined in the Microsoft article "XADM: Bcc Information Is Lost for Journaled Messages in Exchange 2000" (http://support.microsoft.com/?kbid=810999) to configure the journaling of Bcc recipients, you must either remove the JournalBCC registry subkey, or at least set the value to 0, if you want to upgrade to envelope journaling. The JournalBCC registry subkey is incompatible with envelope journaling. After you delete or modify the registry subkey, you should restart the Exchange Store process and the SMTP process before you enable envelope journaling.

You don't need to restart any services after you enable (or, for that matter, disable) envelope journaling. However, because an AD attribute controls the configuration of envelope journaling, not all servers that have journal-enabled databases will use envelope journaling until the new heuristics attribute value has successfully replicated through your organization. Actually, when you enable even message journaling for a given Exchange database, the msExchMessageJournalRecipient AD attribute of that database is updated with the value of the journal mailbox's distinguished name (DN). This new attribute value must replicate through your organization before journaling will work reliably.

If you run a mixed environment that contains both Exchange 2003 servers and Exchange 2000 servers, you can still enable envelope journaling, but the system will handle DL-expansion–related journaling (which I discuss later) incorrectly unless you apply some updates to your Exchange 2000 servers. Specifically, to operate correctly with envelope journaling, any Exchange 2000 servers must be running at least Exchange 2000 SP3, with the August 2004 Exchange 2000 Server Post-Service Pack 3 Update Rollup. (For information about how to obtain the Update Rollup, see the Microsoft article "Update Rollup for Exchange 2000" at http://www.microsoft.com/downloads/details.aspx?familyid=363a57a4-8bed-4bbb-bbe4-abc11ab04611&displaylang=en.) In fact, even if you have no Exchange 2003 servers and you exclusively run Exchange 2000 SP3 servers with the Update Rollup, you can still use the Exejcfg tool and fully implement envelope journaling in your organization.

Envelope-Journaled Messages
A message that envelope journaling processes consists of two parts: a journal report and the original message as an attachment. The journal report identifies the message originator and the message ID and contains the definitive list of the message's recipients, including all To and Cc recipients, recipients expanded from a DL, and any Bcc recipients. (It's worth pointing out that even hidden members of a DL are journaled.) The journal report contains both display-name properties for all addresses and actual SMTP addresses (as determined from the P1 headers). Note that the header on the journal message reflects that of the original message so that DLs appear intact and Bcc recipients aren't displayed. Figure 3 shows part of a journal message that clearly shows the journal report for a message sent to a DL (which contains only three members) and a Bcc recipient.

With an envelope-journaled message, the original message is retained as an attachment to the journal message. The journal message includes information about all message recipients, including mailboxes, public folder recipients, contacts, ad hoc recipients (i.e., SMTP addresses that don't exist in the Global Address List—GAL), alternate recipients, and members of DLs, including query-based DLs that have been expanded locally, which I discuss in a moment.

Envelope journaling also supports message reports—that is, delivery notifications, nondelivery notifications, read receipts, or out-of-office notifications. Exchange captures these types of messages and places them in the journal mailbox. Figure 4 shows an example of a nondelivery notification relating to a recipient from the message in Figure 3.

Envelope-Journaling Mechanics
The mechanics of envelope journaling and message delivery are straightforward. If a journaling-enabled database sends or receives a message, the Exchange categorizer sends a copy of that message to the journal mailbox. The Exchange categorizer makes the decision to capture the message, determining that a message should be journaled if the database has been marked for journaling (i.e., the msExchMessageJournalRecipient on the database isn't null). If the message is to be journaled, the Exchange categorizer bifurcates the message (i.e., forks it into two identical parts) and queues the copy to the journal mailbox.

The Exchange categorizer adds two Messaging API (MAPI) properties to both the original message and the journal message: It sets the value ExJournalData on the PR_JOURNAL_IDENTIFIER property, and it adds any journal recipients to the PR_JOURNAL_RECIPIENT_LIST. These MAPI properties reside in the XEXCH50 BLOB to be used for message delivery over SMTP. Note that the XEXCH50 SMTP extension is specific to Exchange.

When the journal mailbox receives the journal message, that journal message can't itself be journaled (if the database that holds the journal mailbox is marked for journaling) because the Exchange categorizer inspects the message and determines that it's a journal message, according to the journal properties.

In the simplest of examples, suppose a message's originator is on a journaling-enabled database and the recipient is on a nonjournaling-enabled database. (For the sake of illustration, I'm referring to databases on separate Exchange servers.) Web Figure 1 (http://www.windowsitpro.com/microsoftexchangeoutlook, InstantDoc ID 45644) shows the message flow.

In more complex environments, in which journaling is enabled on both the originator's database and the recipient's database, the Exchange categorizer—when first invoked at the originator—will journal the message for both the originator and the recipient. If the recipient's database has journaling enabled to a journal mailbox that's distinct from the originator's journal mailbox, the Exchange categorizer at its first invocation will journal the message to both journal mailboxes, as Web Figure 2 shows.

The message is journaled to both originator and recipient journal mailboxes at the Exchange categorizer's first invocation because the Exchange categorizer inspects each recipient on the message, determines to which databases the message must be delivered, then inspects the msExchMessageJournalRecipient attribute for each of those databases to determine whether the message is to be journaled. Because the journal messages go to both journal mailboxes, Exchange updates the original message's PR_JOURNAL_RECIPIENT_LIST property to indicate that a journal message has been sent to both journal mailboxes. Therefore, when the Exchange categorizer processes the message at the recipient database, the message isn't rejournaled.

DL Expansion
When a server sends a message with a DL, it must expand the DL to determine its membership. If the originating server can expand the DL, Exchange can determine the membership and handle the DL members just like any other recipients. However, if the originating server can't expand the DL, the dedicated DL-expansion server must perform some of the journaling operation. In the example that Web Figure 3 shows, the originator sends a message to a DL that can be expanded only on the recipient's server. In this example, the single mailbox that represents the membership of the DL is hosted on a database that's enabled for journaling.

The following sequence of events takes place. A journal message goes to Journal Database 1 to journal the fact that the message's originator sent a message to a DL. Figure 5 shows an example of this journal message, which shows only the name of the DL—not its membership (which can't yet be determined). The originator sends the actual message to the recipient server, on which the DL is expanded. The DL-expansion server expands the DL and sends a journal message back to Journal Database 1 to display the recipients of the DL and to complement the first journal message sent. Figure 6 shows an example of this journal message.

Because the delivery of the message to the recipient must be journaled, the recipient server also sends a journal message to the dedicated journal mailbox for the recipient's database. Even if the DL-expansion server hosted no databases (and therefore no mailboxes), it would still send the journal message back to Journal Database 1 because it will inspect the message, detect the journal properties, and realize that the actual message recipients haven't been correctly journaled.

Note that in the previous examples, Exchange sends multiple distinct journal messages for just one originally sent message, but all journal messages refer to the same original message and reflect this in the Message ID that's referenced in the journal report.

Comprehensive Method
By catering to DL expansion, Bcc recipients, and message notifications, Exchange envelope journaling is clearly a more comprehensive journaling method than simple message journaling. Envelope journaling also becomes more complex as the distribution of recipients across multiple servers increases with any given message. Accordingly, servers might generate many journal messages for just one originally sent message. And those journal messages might come from a variety of different servers representing the source server, DL-expansion servers, and recipient servers. (Message journaling behaves the same way.) Nevertheless, for any given email message sent by a user in the Exchange organization, you might need to analyze multiple journal messages across multiple journal mailboxes to determine everyone who received the message. Furthermore, for any given email message that a user receives, the comprehensive journal messages stored in journal mailboxes also let you determine who sent that message and who else received it.

Don't construe Exchange's journaling capability as a solution to compliance requirements. Exchange journaling merely provides some basic functionality for the capture and immediate storage of messages sent within an organization. And as I've shown, the sheer volume of data associated with those messages can be staggering, with multiple messages often generated in response to one source message. If you want to make sense of the data you capture, it's imperative that you implement third-party data-management, archiving, and compliance solutions that offer indexing and retrieval capabilities alongside Exchange journaling functionality.