New report slams Office 365 compliance features unfairly

Compliance is a pretty big deal for many companies, especially those who operate in regulated industries. Almost every jurisdiction mandates that companies preserve certain records related to their business dealings. Given the widespread use of Exchange and SharePoint for communications, it should come as no surprise that Microsoft has invested heavily in building compliance features to support their use in business operations.

At least, that statement is true in one aspect. Microsoft has delivered a wide range of compliance features in Exchange 2010 and subsequent releases, including Exchange Online running inside Office 365. The record is less impressive for SharePoint and Yammer, a point hammered home in a July 2015 white paper by Osterman Research, sponsored by Knowledge Vault, Good Technology, GWAVA, KeepIT, Mimecast, and Smarsh.

Any sponsored report has to be treated with a certain care. The sponsor has a view they wish to share with the market and the white paper serves as a vehicle to create a case to support that view. In this instance, vendors like Knowledge Vault have some reporting and auditing capabilities that it would like Office 365 tenants to consider if they’re in the market for compliance tools. Ideally, the text should lay out the case for supplementing whatever Microsoft provides inside Office 365 with third party software so that the reader sees the value contained in what the sponsor has to offer.

The document eventually landed on my desk through the efforts of some very persistent PR. It’s based on a relatively small survey sample of between 128 and 186 “decision makes and/or influencers” drawn from mid-sized and large organizations. Personally, I think this sample size is too small to draw any conclusions, especially when only 25% of those surveyed profess themselves fairly or very knowledgeable about the compliance features built into Office 365, possibly because only 33% use Exchange Online. This is an incredibly low percentage based on my experience as almost every Office 365 tenant I know uses email. Let’s dive into some of the findings to see how its content stacks up.

First, I think that the report highlights some issues that Microsoft still has to deal with when it comes to compliance within Office 365. Even though SharePoint Online has recently gained support for Data Loss Prevention policies, preservation policies, and document deletion policies (all managed through the new Compliance Center console) and has had an eDiscovery Center with the ability to search and hold across sites and mailboxes since SharePoint 2013, it’s true that SharePoint Online can’t archive documents because no equivalent of an archive mailbox exists. On the other hand, SharePoint documents can now be preserved in-place and can also be aged out automatically after a set period, so SharePoint is absorbing some compliance features originally debuted in Exchange.

Neither can Yammer conversations be preserved through retention policies or other means. The same is true for public folder content, which is only now slowly gaining some compliance capabilities.

It’s also true that Microsoft only deals with Office 365 data when it comes to compliance. Still, the current effort to ingest as much data from other sources into Office 365, including corporate communications such as Facebook and Twitter feeds and Jabber and BlackBerry IM, goes some way to addressing that problem, especially if third parties get on board to create more connectors for the Office 365 Import Service.

Another criticism leveled is that Microsoft delivers “good enough” compliance features. The report acknowledges that Office 365 has to service hundreds of millions of users, amounting to some 1.2 million tenants. A specific compliance requirement for one company might therefore not be found inside Office 365, especially if that requirement is specific to a certain industry or country. In any case, the success of Exchange and SharePoint in the on-premises arena is underpinned by an ecosystem of third party software that fill the gaps left by Microsoft.

And while gaps do exist, Microsoft can argue that they are in the process of building out their compliance suite, that the most fundamental features are present, and that others will come in time. In addition, Microsoft can also point to the work that they are doing to move away from application-specific functionality to policy-driven capabilities that can be used to preserve, remove, or find data across multiple applications.

Like any report, the text represents a certain snapshot in time. Those new Compliance Center capabilities seem to be overlooked, so there’s no mention of the ability of compliance searches to address the acknowledged performance problem with very large (more than 10,000 mailboxes) eDiscovery searches. My sources at Microsoft tell me that a compliance search is capable of handling hundreds of thousands of mailboxes and has done so for some of the largest Office 365 tenants. I think that the “wizards” available in the Compliance Center also address the criticism that “Many tasks in Office 365 require familiarity with and use of PowerShell to complete”, assuming that this point relates to compliance functionality.

The new unified auditing subsystem supported by the Management Activity API, designed to extract audit events from all of the Office 365 applications (SharePoint is already supported, Exchange will be soon, and the other applications thereafter) is also overlooked, and the report curiously worries about the 150 MB limit for sending large files, which apparently might cause employees to resort to consumer grade file sharing. The point here is that if you need to send large attachments that don’t fit under Exchange’s limit, they can be shared using OneDrive for Business, which is completely ignored in the report. Indeed, Microsoft is busy changing Outlook and Outlook Web App to encourage people to use “smart attachments” to address this issue. Changing user behavior is not something done overnight and it’s true that people will continue to use whatever they are accustomed to in order to get work done. In the case of DropBox, at least whatever documents are stored there can be retrieved and brought into Office 365.

There’s also a worry that policy-driven message encryption is not supported within Office 365. Apart from Exchange transport rules, it is true that there’s no policy-driven framework for applying encryption to email, but a number of other answers do exist. For example, Outlook Protection Rules can be used to apply Information Rights Management (IRM) templates to messages from the desktop on. And while those rules can be overridden (if permitted) by users, IRM templates can also be applied in transport rules. IRM is a royal pain to deploy on-premises, but it is so much easier to set up and manage within Office 365. And you can also use Office 365 Message Encryption to protect confidential email to external recipients, again enforced through transport rules. Users don’t have to do anything to apply encryption to messages as everything is done as email passes through the transport system.

I can’t find a source to back up the assertion that “Microsoft recommends against having more than one In-Place Legal Hold on a given mailbox at any one time, although Office 365 can handle up to five concurrently (with consequential impacts on performance).” My understanding of the situation is that after five holds are placed on a mailbox, Exchange places the entire mailbox on hold so as to avoid problems that might arise when resolving five different queries. I also understand that this is an implementation choice and five, six, or seven holds don’t really make much of a difference: it’s just easier to preserve everything when multiple holds are in place. I don’t understand why performance is an issue because Microsoft runs more than sufficient servers inside Office 365 to handle tasks of this nature. Besides, users aren’t impacted because the processing of retained documents occurs within a background mailbox assistant called the Email Lifecycle Assistant, which resolves the queries against items before they are removed from a mailbox. Asserting that this is a performance issue is curious.

Also, the assertion that “An eDiscovery search is likely to produce different results each time it is executed. In Office 365, the search query cannot be saved for repeated execution…” seems curious and is not borne out by my experience. Different results can be generated by a search, but that simply reflects the content of the Search Foundation indexes that are being constantly updated as new items arrive or items are removed. Search queries are saved in a search and can be repeated ad nausem.

I could go on but the point is made. I cannot make the case that Office 365 is perfect when it comes to compliance because it is not. Gaps exist as described in the report. It’s also fair criticism to say that the bulk of compliance features are only available to tenants running the E3 and E4 plans (and the new E5 plan when available) and their academic/government equivalents. A case can be made that more compliance features should be available to all tenants, but then you can argue that the folks running small Office 365 tenants probably don’t care too much about compliance because their business doesn’t require these features.

The report says that 59% of those surveyed expect compliance to get worse under Office 365. Well, I guess it all comes down to how a question is framed and the people who are asked the question. My view is that there are more compliance features to be exploited in Office 365 than are available in on-premises software, so I have a problem understanding how the issue could become worse, especially if time is freed up for administrators because they don’t have to do mundane server management tasks any more.

“The Role of Third-Party Tools for Office 365 Compliance” is a curate’s egg: good in parts. It contains value in many points that should be considered by those charged with oversight of compliance within an organization. The development cadence within Office 365 and the speed at which new features appear (not all of which are fully baked when revealed to First Release tenants) makes it difficult to stay up to date in what can be a complex area. Some new functionality is overlooked and some existing functionality is simply ignored. In mitigation, I acknowledge that some time elapses between a report being commissioned and when it appears, and the frenetic pace of change that exists within Office 365 makes it terribly difficult for anything written to remain up-to-date.

Read the paper to see what you can get out of it, but remember, you know your business better than anyone, so take the points presented and put it into context with your knowledge to understand where it adds value for your organization.

Follow Tony @12Knocksinna

[Correction: I originally stated that survey data was used for three reports; Michael Osterman has told me that data was used solely for this report.]