Messaging Security

Learn about 5 ways to encrypt your email communications

Devin Ganger

November 19, 2006

6 Min Read
ITPro Today logo in a gray background | ITPro Today

Microsoft Exchange Server gives you several options for keeping your email message data secure. The options fall into two categories: transport encryption, which protects the connection between two machines and thus protects the contents of the message over that conversation; and message encryption, which protects the message itself independent of the connections it travels over, ensuring that only the recipient can read it.

Transport Encryption


Transport encryption, also known as session encryption, protects the protocol over which the message data is transmitted. The three main kinds of transport encryption are: Encrypted Messaging API (MAPI), Secure Sockets Layer (SSL) or Transport Layer Security (TLS), and IPsec.

If you want to secure sessions between Microsoft Outlook and Exchange, enable encryption of the MAPI protocol. MAPI uses remote procedure call (RPC) between the client and server. By default, the RPC streams are unencrypted, but you can enable 128-bit RPC encryption in the Outlook user profile. Be aware that this setting provides a relatively weak level of encryption and only protects the MAPI traffic between Exchange and Outlook. It does nothing to protect email messages that go beyond the sender's mailbox server.

SSL and TLS are similar protocols in that they provide application-level encryption. The main difference is that you must establish SSL communications on a different port from their unencrypted counterpart, while you can establish TLS over an initially unsecure connection. Exchange currently supports TLS only over SMTP connections but supports SSL over POP3, IMAP, Network News Transfer Protocol (NNTP), and HTTP. ( Exchange's TLS implementation isn't fully opportunistic; you can't handle both secured and unsecured connections from the same virtual server.)

IPsec is a set of IP extensions that provide the ability to authenticate and encrypt IP communications. IPsec has been part of Windows since Windows 2000. Although IPsec is more complex to enable and manage than other options, it also provides more flexibility. Because IPsec is an OS feature, it can protect any application (unlike SSL and TLS, which require that an application support the standards). With Windows IPsec, you need only create the proper Group Policy Objects (GPOs) and attach them to the proper containers in Active Directory (AD) to establish the necessary IPsec filters and behavior. IPsec is therefore suitable for protecting connections between front-end and back-end servers.

If your email message data travels over multiple hops, each hop must be secured or you potentially expose the data to unauthorized parties. If you do secure each hop, the message will be secure while traveling over the wire but will still be unencrypted on each intermediate system between the sender and recipient.

For more information about how to secure the various protocols used in Exchange (and where to use them), see the resources in the Learning Path box, including "Front-End and Back-End Server Topology Guide for Exchange Server 2003 and Exchange 2000 Server" in the Exchange Server 2003 Technical Documentation Library at http://www.microsoft.com/ exchange/library.

Message Encryption


By contrast, the technique of message encryption sounds simple: Encode the message data with a key that only the recipient knows. No matter how the message gets there, you can be confident that only the recipient can read it. The two best known standards for message-level encryption are pretty good privacy (PGP) and Secure MIME (S/MIME).

PGP. PGP and the open-source alternative GNU Privacy Guard (GPG) provide a more ad hoc approach to implementing message security than S/MIME. Unlike S/MIME, PGP and GPG don't require a central public key infrastructure (PKI); users manage their own list of digital certificates (known as their keyring). Because of this feature, users must have a third-party plug-in to use PGP with Outlook. And, as far as I know, you can't validate PGP-signed messages (or view PGP-encrypted messages) in Microsoft Outlook Web Access (OWA) at all. In my experience, PGP is great for single users, but it doesn't scale well to the enterprise level.

S/MIME. S/MIME is an Internet Engineering Task Force (IETF)approved method for formatting and exchanging digitally signed and encrypted electronic messages. S/MIME requires a healthy PKI for deployment. The Exchange/Outlook implementation works in tandem with the certificate authority (CA) and AD functionality in Windows to let users easily enroll and gain their own certificates for securing their email messages. Outlook can then retrieve the necessary recipient certificates for other users in the organization. If you want to use S/MIME with users outside your organization, you need to ensure that your PKI is properly configured, a topic beyond the scope of this article.

S/MIME works as follows (PGP works similarly):

  1. The sender composes an email message in his or her mail client.

  2. As the email message is submitted, the message is encrypted and signed according to a specific set of public and private keys (the recipient's public key and the sender's private key).

  3. The message is routed through intermediate systems. It's impervious to inspection by outside parties; tampering and alteration attempts invalidate the digital signature.

  4. The recipient receives the email message. The recipient's client automatically inspects the digital signature to ensure validity, then applies a private key to decrypt the message.

Does this process sound too good to be true? S/MIME does indeed have two major catches. First, as I mentioned previously, S/MIME requires a working PKI to manage the public and private keys for recipients and senders. Exchange 2003 and Exchange 2000 provide, in combination with Windows 2003 and Win2K, the necessary capabilities to create a digital certificate PKI infrastructure suitable for use with S/MIME. This PKI infrastructure integrates with AD, letting users easily manage their own private keys and retrieve public keys for other users in the organization. The Outlook client supports S/MIME, and the Exchange 2003 version of OWA extends S/MIME support to OWA users.

Second, email messages are encrypted before they're submitted to the Exchange message store service. In other words, they travel across the wire, through all intermediate servers, and are stored in the mailbox in encrypted form. This approach disallows all sorts of useful functionality, such as the ability to scan message body content on behalf of server-side rules. S/MIME also interferes with the application of message hygiene measures and message retention and archival policies. Because the message is encrypted before it's ever submitted to an Exchange server, the Exchange organization can't decrypt the message to inspect it.

To use S/MIME, you need an S/MIME-aware software solution .These applications typically can access all user certificates. They scan email messages before they leave your organization and use the appropriate certificate to secure messages as required.

You need to be able to audit the S/MIME email-encryption process. You must also have good controls in place to manage access to the certificate repository because it will become a natural target for attacks. Finally, you need to ensure that your backup, recovery, and archival processes properly handle the complications your message security introduces. For example, you must have backup copies of the certificates, be able to rebuild your PKI, and archive old and expired certificates used for messages still in your archival system.

For more information about deploying S/MIME, see the resources in the Learning Path box, including "Message Security Guide for Exchange Server 2003" in the Exchange Server 2003 Technical Documentation Library.

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like