Best Practices for Managing Big Content: SharePoint

by Matt Maunder
Content is growing at such unprecedented rates, it seems every previous estimate of total content is rendered obsolete.  In 2011, IDC estimated that by 2020 total content would be approximately 36 zettabytes (ZB).  Within a year, it adjusted the 2020 estimate up to 40 ZB.  That’s a nearly-unfathomable 40 billion terabytes. 

This growth is partly due to the proliferation of tablets and smartphones entering the consumer and enterprise worlds, as well as the pervasiveness of technologies such as Microsoft SharePoint, which make it far easier to create, collaborate on, and store complex content. 

We’re no longer simply talking about big data in the forms of rows and columns in a database—it's also rich content such as videos, images, presentations, complex documents and engineering files such as blueprints and renderings. 

Chances are that your enterprise content is stored in different locations including content management systems, file shares, email, public folders, cloud storage and local disks.  With content so disparately located and growing at a pace far beyond your employee or even business growth, how do you effectively manage it? 

The Power of SharePoint

The solution for many organizations is to consolidate content into one platform--SharePoint.  SharePoint is a powerful product--the Swiss Army Knife of the ECM world--so it’s no surprise that organizations ranging from small businesses to enterprises are using SharePoint.  Many third-party solutions can help consolidate the content from these various disparate platforms into SharePoint to realize this vision. 

The ability to have a single place to access and manage different types of content, view intelligence, and create composite applications is powerful.  It means you only need to manage one content management platform and, as a result, run a more efficient infrastructure as a result.

The Importance of Governance and User Education

Although centralized content is beneficial, your users will find it only if it's accessible, searchable, and relevant. Frequently users don't want to add metadata and properly classify content, as these are "extra steps," so education to ensure relevance and searchability is extremely important.  Organizations need solutions that make it easier to ensure content is stored in the right location, but at the same time can be easily moved, managed, and classified as the content or need for that content changes. 

While SharePoint's growth can be managed, it's more difficult for content owners to manage large amounts of content, and thus we see the “SharePoint sprawl” phenomenon. SharePoint sprawl occurs when content growth outpaces the ability to manage it. This leads to content being located in many different places and negatively affects users’ productivity. A proper governance framework is critical to ensure the platform's growth is controlled and managed.

Storing SharePoint Content

At the same time, we need to consider what effects storing large amounts of content in SharePoint can have on scalability, performance, and cost.  Even though SharePoint has evolved significantly since the early days of Microsoft Content Management Server, still, since 2003, SharePoint has been dependent on SQL Server to store content.  

It is widely acknowledged that SQL Server isn't the most efficient method for storing unstructured content, known as BLOBs. As content grows, the database becomes more fragmented and ultimately the cost of storage increases, as SQL demands the use of high performing, but expensive, tier 1 storage. 

Further, as content and cost goes up, SharePoint performance can go down significantly.  As data is transferred between SQL Server and the SharePoint servers, and ultimately the user, SQL Server can become a performance bottleneck. 

Storage Strategy: Remote BLOB Storage

This is where you should consider other approaches to ensure you have a robust, efficient and scalable platform while reducing total cost of ownership.  For example, Remote BLOB Storage (RBS) is a fantastic mechanism to combat this, enabling you to externalize up to 95 percent of the content in a SharePoint Content Database onto different storage tiers. 

Externalizing the content out of the database has a number of positive effects.  First of all, you can immediately reduce the cost of storage, as you can take advantage of cheaper tiers of storage, whether it is NAS, JBOD or even the cloud.  Further, reducing the content load on SQL can give SharePoint a dramatic performance boost, and means SQL backups become much lighter. 

There are a variety of ways of adopting RBS, but a key element to consider is the ability to utilize more advanced concepts like hierarchical storage management, enabling you to control where the content is stored based upon its lifecycle. 

For example, if content is old or infrequently used and edited, why store it in more expensive storage tiers when it is perfectly acceptable to store it in cheaper, less performing storage? 

Consider what happens if content is tagged with certain metadata or is uploaded to a certain area of SharePoint considered confidential--perhaps this content should be siloed into a separate, or even encrypted tier.  Adopting this kind of approach can then drive the right behavior to make sure content is properly classified and maintained.

Ultimately, RBS enables you to develop and implement a far more dynamic storage strategy, rather than be forced to use the storage infrastructure SQL Server must use.

Gaining Control of Content and SharePoint

Content will continue to grow, and content management platforms such as SharePoint will evolve further as the de-facto standard for managing it.  You need to effectively scale as your business evolves, while at the same time ensuring your costs don't spiral out of control. 

Matt Maunder has been with Metalogix since 2011 and is currently a Senior Solutions Engineer working with EMEA customers to help their SharePoint strategies and maximizing their SharePoint investment. He engages frequently with SharePoint partners and jointly works with customers on new and existing SharePoint projects. Prior to joining Metalogix, Matt worked for 4 years as a Technology Strategist at Microsoft UK, working with enterprise customers to develop their IT strategy around the Microsoft solution stack, including SharePoint and Office.