The woes of Exchange mailbox auditing

Mailbox auditing seems like a pretty good thing to have in an email product, if only to answer the eternal question of who deleted a particular message from the mailbox. Exchange 2010 was the first version of the product to include mailbox auditing. The sad news is that not much has changed since in either the cloud or on-premises product, which is both worrying and not particularly good when considered in the context of Microsoft’s overall compliance strategy.

Details of how to enable and use mailbox auditing can be found in an article I wrote in 2013. The same problem observed then where “audit items do not show up in searches straightaway” persists today in both Exchange Online and the latest versions of the on-premises product.

The odd thing is if you use the Get-MailboxFolderStatistics cmdlet to examine the number of items in the Audits folder (under Recoverable Items), you can see that audit events are available pretty soon after a user takes an auditable action. At that point, my suspicions were that the Search-MailboxAuditLog cmdlet was responsible for generating buggy results. I’m even surer of this now, especially when looking for recent audit events. Leaving things alone for a couple of hours appears to generate more reliable results.

Which brings me to a breathless press release that I received from LOGbinder (a security incident and event management or SIEM ISV) on December 17 to inform the world that “Microsoft Exchange Server query commands produce results that are unpredictable and incomplete when auditing all mailboxes for activity less than 24 hours old”. LOGbinder went on to say that they had informed Microsoft about the bug and that “Microsoft suggested a work-around of setting mailbox audit search delay of 24 hours.”

According to LOGbinder, “it is unacceptable to wait 24-hours before viewing your audit logs. By that time, your entire CEO’s mailbox could be downloaded with the culprit firmly ensconced in a foreign embassy!” Quite.

Despite the vision painted of executive mailboxes being threatened by foreign diplomats, the person who sent the release to me seemed quite perturbed when I wasn’t too surprised by the news, perhaps because the fact that Search-MailboxAuditLog wasn’t so good at finding what it should has been discussed at conferences such as IT/DEV Connections in the past.

In any case, a blog post written by LOGbinder’s CTO, Randy Franklin Smith, makes the case that the “bug introduces a huge business, compliance and security impact”. I think there’s a touch of hyperbole here because the bug has been around for six years now, but you should make your own mind up on the topic.

I said earlier that not much had changed in mailbox auditing. Well, a little bit has changed in Office 365 where mailbox auditing is available in the unvarnished state available to on-premises administrators and in the newly-refreshed Office 365 Activity Report, accessed through the Compliance Center (soon to be renamed the Office 365 Protection Center). I should also note that Office 365 now allows owner actions to be audited. This was always possible on-premises but not in the cloud, so whatever barrier existed to prevent the recording of owner-initiated events was recently removed.

The Office 365 Activity Report draws from a data mart of events extracted from the various workloads that run within the service. Exchange mailbox audit events is one source. The new version of the Activity Report allows you to view events relating to SharePoint, OneDrive for Business, Azure Active Directory, and Exchange (for now) and export the event details to a CSV file if required. However, the current version available to First Release tenants suffers from the fact that it depends on the data provided in mailbox audit events – if the events don’t contain the information or if some events are missing (because they haven’t yet been extracted) then you end up with an incomplete picture.

Microsoft is well aware that things are not as good as they should be when it comes to mailbox auditing. The Unified Auditing architecture created for Office 365 is supposed to deliver a real-time experience, but it doesn’t deliver on this promise, at least not for Exchange mailbox audit events.

What does all this tell us? Well, the first thing is not to depend on mailbox audit event searches for events logged in the recent past. If you want to search, let a little time go by before you start looking. The second is to hope that the work Microsoft is doing around the Unified Auditing system for Office 365 will cure the problems that afflict mailbox auditing.

Mailbox auditing is a very useful feature and one that people expect to work. It would be nice if it did in a predictable and reliable manner all the time and not just when it wants to.

Follow Tony @12Knocksinna