EDRM and SharePoint: Designing and Building a Compliant Platform

The amount of information generated by organizations is growing at a phenomenal rate. With factors such as increased legislation and penalties, many organizations want to leverage their information assets, both physical and digital, to their full advantage by increasing speed of access, improving classification schemes, and assuring information quality and trustworthiness. To do so, organizations are adopting toolsets such as Electronic Document and Records Management (EDRM) applications, Microsoft SharePoint, and other document management tools for creating a central trusted repository, which is a key goal of records managers.

There are many roadblocks to attaining that goal, such as executive sponsorship and enforcement, organizational alignment, confusion over toolsets, lack of a consistent and user-relevant classification scheme, poor usability, and low user adoption. Most organizations have islands of EDRM systems in various states of deployment that are generally not delivering the value they promised. That's why having a good planning and deployment strategy in place is crucial. 

Planning your Records Management Solution

Planning a records management solution is a daunting task. To be successful, you need a multidiscipline team because designing the system requires multiple skills and diverse knowledge (e.g., communications, governance, information architecture, records management, data migration, security, EDRM system architecture). A good place to start the planning process is to read ISO 15489 and the DIRKs methodology. (See the Related Resources Box.) Here are some key points that you need to consider when planning your EDRM rollout:

Records Center 2007 Overview

Although Microsoft has released SharePoint 2010, there is merit in covering SharePoint 2007 technologies because many organizations won't upgrade for another year or so. SharePoint 2007 has a Site Template called Records Center. It has functionality such as Holds, which lets you place holds on retention schedules, and Auditing, which logs events and operations against the record. It also contains some specialized libraries and content type routing configuration that allows you to route records by content type. 

For example, you could create a content type called Contracts and a library for the content type. You would then create a routing rule in Records Center that would route the content type to the Contracts library. You could then apply Information Management policy (i.e., how to handle records) to the library. The final step in the process is to configure the farm to be aware of the Records Center by adding the link to the Records Center Web Service (e.g., http://server_name/_vti_bin/officialfile.asmx). To do so, select External Service Connections and enter the URL to your records center web service in the Connect to a Records Repository field. Upon completion, the option to route records within document libraries will now appear in the item's edit/context menu as Figure 1 shows. Note that any work you do should be done in a sandbox environment, which is usually a virtual machine (VM) running on your laptop.

Create a Records Center

This section outlines the basic process for creating a Records Center in Microsoft Office SharePoint Server (MOSS) 2007. For more detailed information, see the Microsoft TechNet article "Create a site collection (Office SharePoint Server)". To create a Records Center, follow these steps:

You now have a functional Records Center site. If desired, you can add the Records Site to your Portals navigation by editing the Portals navigation menu and adding a tab for the new Records Center site.

Records Center 2010 Overview

In SharePoint 2010, the Records Center has a slick new UI and some additional features that improve usability and management. For those who evaluated SharePoint 2007 and held off, SharePoint 2010 offers compelling new features and enhancements. Here is an overview of some of the new functionality.

      In-place records management. A new industry trend is the idea of in-place records management. Specifically, instead of a central repository of documents that requires a routing service or manual movement of files, records are managed where they are. The documents stay in the current location and are classified as business records. This approach avoids the laborious migrations associated with EDRM projects.

      Multistage retention. Retention policies can have multiple stages, allowing you to specify the entire document lifecycle as one policy (e.g., review Contracts every year, and delete after seven years).

      Folders. Another interesting note is the new role of folders in SharePoint 2010. In SharePoint 2007, folders were the recommended option for grouping files by security access or to be avoided. With SharePoint 2010, folders can act as true parents to any child objects below it. Now, metadata is set at the folder level so that the child objects can inherit that information.

      Document sets. Document sets enable users to collaborate on related documents without having to create a new document library or site. Their purpose is to help organize documents and enable collaboration on documents tagged with similar metadata.

      Persistent document IDs. The document ID provides absolute reference to objects, regardless of filename changes or document moves.

      Content organizer. The routing rules from SharePoint 2007 have been replaced by the content organizer, a new SharePoint feature available in all document libraries. The content organizer routes documents to the right folder based on content types and any other metadata that you require.

      Compliance details. This is a new feature that adds a Compliance Details option to the context menu. It allows users to check out the relevant settings that have been applied to a specific business record. The feature also enables administrators to make sure that specific documents are inheriting the right policies and retention settings.

      Submit a record. This is a basic feature that adds some simple usability to the site.

For more information, read the article “8 Reasons SharePoint 2010 Looks Like a True ECM System”.

Creating a Records Center in SharePoint 2010

Note that SharePoint 2010 enables administrators to create multiple connections to Record Centers per farm, and to specify more detailed parameters about the behavior of those connections. Another enhancement is the ability to move an item to the Records Center, or to move an item, but leave a link in the original location.

Next you need to create content organization rules. You add rules by following these steps:

It's important to note that the source and destination libraries (i.e., document libraries configured for inplace records mgmt) will require the same content types to used. For example, if you have a Contracts Form content type in your source library, you will also need one in your destination library.

In Place Records Management

To take advantage of In Place Records management, you must first activate the feature for the site collection, as Figure 2 shows. If you have a site collection configured in your sandbox environment, go to Site Actions and activate the feature. Otherwise, create a site collection (e.g., Team Collaboration), then activate it.

If you want items within the libraries to be automatically flagged as records (e.g., a Contracts library), choose the library in your site that you want to enable to support in place records management, and click Library Settings. Enable Automatic Declaration. Now, all documents added to the library are automatically flagged as records. Note that after you activate the feature and configure the library, a new button, Declare Records, appears in the Libraries ribbon. You can read more in the TechNet articles "Records management overview (SharePoint Server 2010)" and "Designing for in-place records management".

Third-Party EDRM

Butler Group released a review of the top records management vendors and their products in late 2008. EMC Documentum, IBM FileNet, and OpenText were the notable top players. HP TRIM is classified as Consider, and SharePoint 2007 is classified as Explore.

An important pattern to note is the rise of SharePoint. It’s disrupting the EDRM market as well as some others such as Web 2.0. With its high cohesion with Office, its ease of use (generally a high rate of adoption), and its low cost from a licensing perspective, SharePoint is a good option for more and more companies. According to Butler Group, “SharePoint is designed to help organizations manage content while enabling information workers to communicate and collaborate electronically.” EDRM vendors have made attempts to integrate with SharePoint using simple web parts that expose the File Plan and provide varying levels of search integration. The web parts typically include:

Document library. This web part enables Site Administrators to expose the EDRM File Plan as a document library. You need to specify the area (Container/Class) of the File Plan to expose it. This web part can make the provisioning process more difficult because of the skills required to configure it when provisioning sites; therefore, adopt with caution.

The Edit/Context Menu will include the “Move Document Feature” option. This web part adds a feature to the Edit/Context Menu that enables users to select a document to be moved to the EDRM File Plan. It generally exposes an ASPX page that requests record type and required metadata before sending to the EDRM.

Search protocol handler. This web part is installed on the Index Server and enables the server to crawl third-party applications such as an EDRM. For SharePoint to display combined search results from a content source, it must understand how to access, crawl, and return results. This is the job of the handler. I've found handlers to perform slowly and sometimes hang the indexer. Adopt with caution.

Federated search web part. If vendors don't have the time or expertise to develop a handler, the federated search web part is available. SharePoint search passes the web part, receives the search criteria from SharePoint search, processes the query, and displays it as a separate results set. I've found these web parts easy to install, and they perform well.

I’ve worked with a few of the EDRM's products web part integration and was not impressed. They don't provide true integration, but instead are more or less a simple gateway to the EDRM via web parts. They don't provide the ability to take advantage of features such as alerts, work flows, and views. Indexing performance is generally slow and doesn’t scale well. There are often stability concerns and provisioning is problematic. Given its almost viral growth rate, SharePoint will continue to disrupt the EDRM market as it did the Portal market.

To the Future

To understand how the way people communicate has changed over the past century, and especially in the past 20 years, think about the communications media that past generations used: letters, phone, and fax and of course face-to-face communication. With these forms of communications came a simpler slow-paced work environment than we experience today.

Today, we use a variety of devices and applications such as smartphones, email, IM, cameras, Facebook, Linked In, VOIP, web conferencing, Twitter, and corporate systems such as email and voicemail.

EDRM is no longer the simple capture of physical and electronic assets such as mail and electronic documents. The scope of EDRM spans many applications and communication media and is far reaching organizationally. Some organizations realize this and have locked down corporate access to Internet applications. But what are the employees communicating from home? Are they leaking company secrets? Are there inappropriate communications about the organization or its people? Are recruiters chasing your staff on recruitment and networking sites?

Tools such as SharePoint technologies and Web 2.0 have changed the EDRM market. SharePoint 2010 is applying pressure to EDRM systems to step up their game regarding in place records management. Records managers and IT managers are thinking of ways to capture communication in applications such as Facebook and Twitter. The price point and usability of SharePoint is pushing EDRM tools out and changing the way companies think of and manage information assets. I predict that standard EDRMs will become a backend to software such as SharePoint. EDRM vendors will improve their integration by creating deep linkages to applications such as SharePoint and Web 2.0 applications that are seamless to the average user. Users won’t even know their documents and communications are being captured as records.