What Does Deleted Really Mean?

Forensic science is a fascinating field, as evidenced by the number of people who eagerly watch forensics-related TV shows (ranging from the old-school 1970s drama "Quincy, ME" to today's "CSI" and its spinoffs) and read books in which forensic evidence figures prominently. Computer forensics is a growing field, as well, and during the next couple of months I'll occasionally write commentaries that explore various aspects of Exchange forensics. First, however, I want to talk about what "deleted" really means. A recent post that Exchange Most Valuable Professional (MVP) Rich Matheisen made to a mailing list sparked my interest in the topic, and I thought it was worth sharing. In his post, Matheisen explained what happens to a message or folder when a user deletes it.

When a user deletes an item, it can pass through three stages. In the first stage, a user deletes an item in the usual way, which moves the item to the mail client's Deleted Items folder (most DAV and IMAP clients are smart enough to use the Outlook-style Deleted Items folder, so we'll include them in this discussion). At this point, the user who deleted the item can easily recover it by opening the Deleted Items folder and dragging the desired item to another folder.

In the second stage, the item is removed from the Deleted Items folder. Matheisen calls this the _deleted_ state to distinguish it from the first deleted stage. Items can reach this stage in a couple of ways. The most common way is for an item that was formerly in Deleted Items folder to be permanently deleted because it hit the end of the deleted-item retention period. Users can also permanently delete (i.e., hard-delete) items themselves, however; in Outlook, holding the Shift key while deleting an item hard-deletes it. Items in this second stage are recoverable if the user has enabled the "Deleted item retention time" option. When it requests folder contents, Outlook can add the EXCHANGE_DELETED_ITEMS flag to display the items that are in this stage, which is how the Recover Deleted Items command works.

Objects that have been _deleted_ are still in the Store, but they clearly can't stay there forever. Items "age out of" the Store when they hit the end of the deleted-item retention period. This strategy works because items that have been _deleted_ have the MAPI PR_DELETED_ON property set, which is how the nightly Exchange maintenance task determines when the item was deleted and when to remove it. Such items are removed from secondary indexes, which has the effect of recovering them from views and is why you don't see these items in their usual folders.

When a _deleted_ item ages out, Exchange removes it from the Store. The nightly maintenance task is responsible for marking the pages that the deleted item formerly used as "free," at which point the Store can reclaim the pages. The next backup might also zero in on them, assuming you have the "Zero deleted pages" option enabled in your backup utility.

The process that Exchange uses to delete mailboxes is slightly different; I'll explain that process in more depth in next week's commentary. In the meantime, remember that deleted messages can stay on your server a lot longer than you might expect, so plan accordingly.