Exchange 2010 MRM: Implementing New Retention Policies

We live in a world of ceaseless information. Our mailboxes are stuffed with messages, and we have little inclination to spend the time and energy to impose any order in our mailboxes. We keep everything until we're forced to free up some mailbox space when our quotas are exceeded. Outlook's automatic archive feature helps by regularly moving items from a mailbox into a PST, but it's a crude instrument because it moves items based on date rather than permitting more granular selection criteria. It also depends on the deployment of a single client, and that’s just not a viable strategy in a world where client choice is the norm.

Microsoft Exchange Server 2007 introduced the messaging records management (MRM) system as its business email strategy to help users comply with regulatory and legal requirements. The concept is simple: automate the retention process so that users have an easy way to control what's in their mailboxes and retain those messages and attachments that are required business records.

To implement this concept, Exchange 2007 MRM uses a set of managed folders that have policies attached to them. The Managed Folder Assistant (MFA) applies the policies attached to the folders. Marked items are kept as long as required; other items are automatically discarded when their retention period expires. (For more information about Exchange 2007's MRM system, see the web-exclusive article "Using MRM to Manage Mailboxes".)

Although the concept is good, in reality Exchange 2007's MRM system is typically ineffective because Exchange users are relatively undisciplined when it comes to filing messages in folders. So, Microsoft changed its tactics to provide a workable implementation of MRM in Exchange 2010.

A New Approach

Instead of using managed folders, Exchange 2010 MRM uses retention tags and policies. (Managed folders persist in Exchange 2010 but only for backward compatibility.)

Retention tags. Exchange 2010 supports three types of retention tags: retention policy tags (RPTs), default policy tags (DPTs), and personal tags. Table 1 describes these tag types.

Retention tags can be applied to any item in any folder to specify what action Exchange should take for the item when its retention period expires. Supported actions include the hard (permanent) or soft (recoverable) deletion of items, moving items to a personal archive, or flagging items for user attention. Retention tags can be placed on items, conversations, or complete folders. They're transferred with items if you move them between folders.

Retention policies. Retention policies group retention tags together so that administrators can apply policies to mailboxes rather than having to assign individual retention tags to folders. Retention tags and policies are organizationwide objects that are stored in Active Directory (AD) and can therefore be applied to any mailbox in the organization after they're created. The MFA is still responsible for checking mailbox contents against the policy and taking the specified action for each item that exceeds its retention period.

Retention polices and tags follow a couple of simple principles. First, Exchange can apply only one retention policy to a mailbox. If you apply a retention policy to a mailbox that already has one, Exchange will overwrite the policy already in place. Second, Exchange can apply only one retention tag to an item. If an item has more than one associated retention tag, Exchange uses two rules to determine which tag to apply:

To use Exchange 2010 MRM, you need to deploy clients that include the necessary intelligence and UI. At the time of this writing, the only clients in this category are Outlook 2010 and Outlook Web Access (OWA) 2010. Outlook 2010's UI provides the richest views of retention policies and tags. OWA 2010 is less capable but still very usable.

Designing a Retention Policy

Many different retention policy tags can exist within an organization, which allows great flexibility in creating appropriate policies for different groups that work within a company. For example, the finance department might want Exchange to permanently delete everything in the Deleted Items folder more than three days old (i.e., the shred principle), whereas other departments might not be concerned if items survive in the Deleted Items folder for 30 days or more. For the finance department members, you can apply a retention policy that includes an RPT that instructs the MFA to permanently remove items from the Deleted Items folder after three days. The same policy might include a personal tag that lets financial department members mark items that have to be archived for audit purposes after a month in their primary mailbox. The MFA will move items with this tag to the archive mailbox when it processes the mailbox.

Before you create a retention policy, you should determine the why, when, and how for the policy:

The design for a retention policy can be captured in a simple table format that makes it clear what tags are included in the policy, their purpose, and the folders to which they apply. Apart from anything else, capturing the design like this makes it easier to communicate the policy to users. Table 2 lays out a sample policy that could be applied to help managers cope with their overloaded mailboxes. Logically, you should have only one RPT for each folder. It would be very confusing to have two RPTs compete within a single folder.

Although you can create and use as many retention policies as you want, the question of long-term supportability arises. A couple of well-designed, logical policies that satisfy the vast majority of requirements will be easier to manage than a mass of granular policies generated to meet the specific needs of each department or other business group. The more policies that exist, the more potential there is to confuse administrators and users alike.

Creating Retention Tags

The Exchange 2010 MRM is sound in terms of both concept and implementation. The only problem is that Microsoft didn't ship a GUI that you can use to create and apply retention tags and policies in the release to manufacturing (RTM) version of Exchange 2010. Microsoft will provide the missing GUI in Exchange 2010 SP1. A beta version of this code was released at TechEd in June 2010, and the final version is expected to ship later this year. Until SP1 comes out, you can use PowerShell to create and apply retention tags and policies.

Let's create the various tags necessary to build the retention policy in Table 2. To create a retention tag, you use the New-RetentionPolicyTag cmdlet. So, for example, to create the RPT for the Inbox, you'd run the command

New-RetentionPolicyTag  -Name 'Manager-Inbox'  -RetentionAction MoveToDeletedItems  -AgeLimitForRetention 30  -Type Inbox  -Comment 'Inbox items are    automatically moved to    Deleted Items after 30 days'  -RetentionEnabled $True

(Although this command wraps here, you'd enter it all on one line in the PowerShell console. The same holds true for the other commands that wrap.) The -Name and -Type parameters specify the tag's name and scope, respectively. The "Possible -Type Values" column in Table 1 lists the other possible values for the -Type parameter.

The -RetentionAction parameter specifies the action to take for items past the retention limit. In this example, the action is MoveToDeletedItems, which tells the MFA to move the item to the Deleted Items folder. The other actions you can specify are:

The -AgeLimitForRetention parameter specifies the retention limit. Exchange uses the date and time when an item (even a modifiable item such as a post) is created as the baseline to calculate the item's age for retention purposes. So, in the case of the Manager-Inbox RPT, the age limit of 30 days means that items are moved to the Deleted Items folder 30 days after they're delivered to the Inbox.

You can create a tag that tells the MFA never to process an item. To do so, you don't set a value for the -AgeLimitForRetention parameter and you set the -RetentionEnabled parameter to $False. If you set a value for the -AgeLimitForRetention parameter, you must always set the -RetentionEnabled parameter to $True.

The MFA ignores items that are older than the retention period when it processes a mailbox. For example, if the retention period for the Inbox is 30 days, the MFA will tag any item aged up to 30 days, take action for items aged 30 days, and ignore any item older than this. This approach might seem strange at first, but it's actually quite logical because you can't have a minus retention period. The upshot is the potential for user confusion because all the most recent items in a folder (up to the retention period) will be tagged while anything past this point will be ignored. Thus, when you implement a retention policy, you do so from a particular point rather than going back to the start of time.

After creating a tag, you can check its properties with the Get-RetentionPolicyTag cmdlet. For example, to the check the Manager-Inbox tag, you'd run the command

Get-RetentionPolicyTag -id 'Manager-Inbox'

Figure 1 shows the output from this command.

The output confirms that the tag covers items of any class (MessageClass: *, which is the default for Exchange 2010) that end up in the Inbox folder. It also confirms that items tagged with this RPT will be moved to the Deleted Items folder after 30 days.

Unlike managed folders, retention tags don't accommodate the notion of item segregation. In other words, you can't build a retention tag that only applies to items of a certain class in a folder (e.g., apply the policy to items of class IPM.Note, but ignore those of class IPM.Contact). Along the same lines, you can't define different actions for different item types, such as moving expired messages to an archive folder while deleting any other type of item. Some observers consider these shortcomings to be a step backward for MRM.

Using the New-RetentionPolicyTag cmdlet, you can create the remaining five tags listed in Table 2. You don't need to do anything different when creating a personal tag (e.g., the Manager-Retain tag) or DPT (e.g., the Manager-General tag).

To check that you have all the required tags in place to build the retention policy, you can execute the command

Get-RetentionPolicyTag |  Format-Table Name, Type,  RetentionAction, RetentionEnabled,  AgeLimitForRetention -AutoSize

Figure 2 shows the output.

Creating the Retention Policy

Now that you've created the six tags required to help managers impose order on their mailboxes, you can create a new retention policy using the New-RetentionPolicy cmdlet. When you create the policy, you also associate its tags by using the -RetentionPolicyTagLinks parameters, as in

New-RetentionPolicy  -Name 'Management retention policy'  -RetentionPolicyTagLinks  'Manager-Inbox', 'Manager-SentItems',  'Manager-Deleted', 'Manager-JunkMail'  'Manager-General', 'Manager-Retain'

You can use the Get-RetentionPolicy cmdlet to examine details of the new retention policy. For this example, you'd run the command

Get-RetentionPolicy  -id 'Management retention policy'

Figure 3 shows the results.

A six-tag policy is a reasonably simple retention policy. Microsoft recommends that you include no more than 10 retention tags in a policy to avoid confusing users and administrators, which is good advice. There might be times when you need to incorporate more tags in a policy to meet specialized business needs. A more sophisticated policy for a department might have separate RPTs for all the default folders, a set of personal tags developed specifically to suit the department's retention needs, and a DPT for everything else.

Applying a Retention Policy

The Set-Mailbox cmdlet is used to apply a retention policy to mailboxes. A mailbox can have only one retention policy, so when you assign a retention policy to a mailbox, the action overwrites any policy that might already be in place. The new policy will be applied to the mailbox the next time the MFA runs.

The command to apply a retention policy might look like

Set-Mailbox -id 'JSmith'  -RetentionPolicy  'Management retention policy'

Exchange will warn you that clients earlier than Outlook 2007 don't support retention policies. No client before Outlook 2010 and OWA 2010 includes the UI necessary to display the retention policy information about items and change the retention tag that’s applied to an item or folder.

If you're setting a policy for a group of users, you can do it in one operation by selecting the mailboxes with the Get-Mailbox cmdlet and piping the results to Set-Mailbox, as in

Get-Mailbox -Filter  \{CustomAttribute7 -eq 'Management'\} |Set-Mailbox -RetentionPolicy  'Management retention policy'

This operation will become a lot easier in Exchange 2010 SP1, when the GUI will be available. You'll be able to select a group of mailboxes and apply a retention policy using Exchange Management Console. EMC will also let you apply a retention policy to an individual mailbox.

To discover the set of mailboxes that have retention policies in place, you can use a command like this:

Get-Mailbox -Filter  \{RetentionPolicy -ne $Null\} |  Format-Table Name, RetentionPolicy -AutoSize

After you begin to deploy retention policies to mailboxes, you need to determine how to integrate the assignment of retention policies with your company's user provisioning process. Exchange doesn't have a default retention policy that can be assigned automatically, so an explicit administrative action is always required to assign a retention policy to a mailbox. As you've just seen, this task isn't difficult to code with PowerShell, but it's something that needs to be considered as part of your deployment plan.

Upgrading From Managed Folders

You can upgrade a managed folder to a retention tag by using the folder as the template for the new tag. The New-RetentionPolicyTag cmdlet's -ManagedFolderToUpgrade parameter makes this possible. Suppose you have a managed folder named Never Delete that acts as a repository for items that users never want to be removed from a mailbox because they're so important. You could argue the case that these items could be stored in an archive mailbox. However, archive mailboxes didn’t exist in Exchange 2007, and it takes time for people to change their behavior. A better alternative might be to create a new retention policy tag based on the Never Delete managed folder by using the command

New-RetentionPolicyTag  -Name 'Mark item to never expire'  -ManagedFolderToUpgrade 'Never Delete'  -Comment 'Tag created from old Never Delete  managed folder'

As you can see, when you use the -ManagedFolderToUpgrade parameter, you don't need to set the -Type, -RetentionAction, -AgeLimitForRetention, and -RetentionEnabled parameters. This information will be retrieved from the specified managed folder.

To complete the process, you have to associate the new tag with a retention policy and apply the retention policy to the users' mailboxes. After this is done, the users will be able to apply the new tag on any item in their mailboxes.

Stayed Tuned for More

The topic of Exchange 2010's retention policies is too big to be covered in just one article. So, I'll cover how to change and remove retention policies, as well as how to help users understand them, in the article "Exchange 2010 MRM: How to Modify and Reduce Help Desk Calls About Retention Policies," which will be online in the near future.