Moving your Public Folders to SharePoint

During the past year, I've worked with several clients that plan to use SharePoint as a replacement for Exchange public folders (and file shares, though not the topic of this article). Given the problems that organizations face with the uncontrollable growth of unstructured data, compliance requirements, and impacts on storage and operations, it’s a painful topic with no "magic bullet" solution.

Is there a rush to replace public folders? Probably not at the moment unless there are organizational factors surfacing that are forcing the change. For the time being, there's no time pressure for organizations to do so, because public folders will continue to exist and be supported until the end of the Microsoft Exchange Server 2007 product life cycle in 2016 or 2017. But there are compelling reasons for adopting a SharePoint-based solution: better presentation, search, and mobile access to name a few. But expectations must be managed carefully because migrating is a complex labor-intensive undertaking that if handled incorrectly can result in significant business interruption. The guidance I offer in this article is a general overview of the process that will help you plan, design, and carryout your migration with as few problems and pitfalls as possible.

The SharePoint Decision
SharePoint is not the solution for every enterprise. Organizations that are considering the move to SharePoint need to consider the following concerns prior to the move:

Approach
Generally, I recommend that companies take a new approach to thinking about data--its classification, storage and retrieval. Organizations must think about information using a fine-grained approach as opposed to the "big bucket" approach of public folders and file shares. You need to consider how the information in your public folders map to your organizations Information architecture. Will you require sites for sales teams or project teams? Once data is migrated, how will you make sure the metadata is entered for each SharePoint Content Type? For example, information contained within public folders might consist of client information, product information, job related information, client information, or corporate. So how do you make sure that information is transferred to the SharePoint site?

Before conducting a migration to SharePoint, you need to make sure that the public folder (and file share) migration is handled according to organizational policy to prevent compliance issues and that you have a well-thought-out destination for the public folder data to prevent disorganization and users not being able to find the data they're looking for. For example, you need to set what data is permitted on public folders. (e.g., Microsoft Office documents but not MP3s), and what’s the policy for removal of violations? You also need to consider how the information will map between public folders and SharePoint Sites and Pages. Figure 1 provides a high-level view of how information (by functionality) maps to SharePoint.

Note that Figure 1 doesn’t depict how information maps specific to taxonomy (detailed Information architecture). In SharePoint, information is displayed in a more categorized and visible manner than public folders. For example, Contacts are placed in a Sites Contacts Web Part; documents are placed in a document library. Tools that migrate data from public folders to SharePoint will help with this classification and the creation of sites, but ensuring information relevance requires a lot of human involvement. Understanding this, your information architecture must address farms, Shared Service providers (SSPs), sites, pages, Web Parts (applications), content types, metadata, labeling, security, and content organization.

Before I dive into the methodology, I assume that you've already completed a SharePoint design that includes a detailed information architecture, system architecture, and operations plan. These steps are crucial for decision support and having a usable and compliant SharePoint navigation and search experience when you’re finished. If you need help planning your SharePoint design, the following resources are available:

The following sections outline a basic migration methodology, including the steps for each phase and hints for dealing with your project, because this project will change the way you handle your organizations data. Although it might seem like an overwhelming task, keep in mind that these steps are for a very large organization with gigabytes and perhaps terabytes of public folder data. You can always modify these guidelines to better suit your own organization, if necessary.

Phase 1: Project Initiation
In this phase, your goal is to build your team and prepare project documentation that will help steer your project effectively.

Establish a governance team. You need to establish a team and decision framework that will help you steer the rough waters ahead. Given that the project will touch just about every business unit, you'll require senior management to help facilitate the project's momentum and to address escalations and key decisions such as scope, resource scheduling, data retirement, prioritization, and business unit buy-in to name a few. The governance team should consist of executives from IT and business units, IT architects, the project management office, and purchasing, in order to be effective. A framework for escalations and making decisions must exist to facilitate the success of your governance program.

Develop a project charter. As with any project that deals with information, scope creep is your enemy because it could increase complexity and result in lengthy project schedules. Agreeing upon scope and priority is difficult especially if you have requirements that cover migration, security, and data cleanup. No one person can tackle public folder migration on their own--it's too complex. Assemble your team (Tiger team) with leads from the Help desk, data center operations, file servers, Storage, Exchange, SharePoint, records management, desktops, security and communications as a starting point. Public folder migration is far reaching and thus will impact the business. Your communication plan must address who, what, and when, and the business units that are to be migrated should be provided with communication early in the process. Expect push back due to concerns about business interruptions.

Phase 2: Requirements
In the requirements phase, your task is to develop a document and supporting material that will define the specific requirements of your organization and lead your team through the process of developing a site design. For example, your requirements might consist of migrating public folder data to SharePoint sites, establishing security guidelines and configuring SharePoint security accordingly, and ensuring compliance with information management policies and information architecture. The requirements document should contain the requirements for the team, tools, methodology and risk plan. Here are some things to remember when developing requirements:

Project Management Office (PMO). The PMO will have some insight to the project's initial scope, deliverables and time lines.

Compliance department. The compliance department might be actively involved in the cause of past audits that have exposed compliance issues, but if not, I suggest you meet with them to develop a list of compliance requirements. This could be a simple list of principles that must be incorporated into a design (e.g., being able to identify documents that reside on public folders that have legal impact, such as contracts).

IT department. You need to consider the IT department requirements for infrastructure and operations. For example, IT will be interested in capacity requirements and operational impacts (Help desk, monitoring, and n-level support). Note that it's important to involve the Help desk people since they'll need to help with support calls once the data migration begins.

Business requirements. Though often avoided by IT departments, working closely with the business early in the process is critical to managing perceptions and determining their specific needs. In many organizations the business has the final say. Winning them over through building rapport and trust is a must even if there is a history. This will help reduce the chance of push back or the project stalling later on during the actual migration. Also, don’t use the business as your testing ground. Use a lab and build a mockup of a business unit for testing.

Quality Assurance (QA). Many organizations have a QA process that can add significant time to your document acceptance process. Don’t forget to factor this into your timeline and documentation plan. Meet with the QA people to understand what you must provide them and when.

Inventory. The first major task for you will be to develop an itemized inventory of the public folders. To be successful, you’ll require a toolset. The toolset must be able to crawl and inventory the public folders and provide robust and customizable reporting. For example, being able to report on the amount of content, content types, content age, and orphaned data will be an asset. The tools you chose must be installed early on in the project so that inventory of the public folders can begin. Several vendors make tools for migrating public folders to SharePoint including Quest Software, Metalogix, and Tzunami.

Analysis. Generally your analysis will focus on the following:

Other projects. Most organizations have several projects underway at the same time. You must plan for this because collisions will occur and dependencies must be understood. Note that a meeting with the PMO will help you understand what projects are planned or underway and who you should begin discussions with.

Phase 3: Design
During the design phase your task is to develop a document and supporting material that will define the specific elements of your design. The design document will contain (depending on your organization's methodology) the design for the approach, methodology and support materials for your migration. This document should have tight linkages to your project requirements document and must address how the requirements listed in the requirements document will be addressed. The document should include the following items:

Tools infrastructure. Hopefully the toolset you use to inventory the public folders also has the ability to migrate the contents to SharePoint sites and log the results. If not, you must assess toolsets based on the requirements document. Your design must incorporate the technical infrastructure required to support the toolsets for the duration of the project. For example, how many servers do you require for the toolset? Do you need a workstation to act as the operator console? Does the toolset require a database? How much storage does the database require? Are agents required on the servers? What are the impacts to network bandwidth? In the case of low bandwidth networks, is an alternate means required? For example, one organization's network between the United States and Australia didn’t support the bandwidth required to migrate data. To address this issue, the server's drives were removed and shipped to the United States and the migration was performed there. If time permitted, perhaps a replication approach might have been a better approach.

Application remediation. Your design must incorporate tools and processes for dealing with applications that rely on the public folder infrastructure. Generally, the tools should be able to spot such applications; otherwise you must have a manual inventory process for collecting such information from the business units. There must also be a central record (e.g., a spreadsheet or database application) of applications to be remediated. Last but not least, your design should include a design for a development and QA environment for recoding and testing rewritten applications.

Public folder to SharePoint mapping. How will public folder data map to your SharePoint environment? Create a form in Excel that lists the mapping of the folders to SharePoint and any specific notes such as exclusions. Note that migration tools will attempt to create sites based on the public folder hierarchy and populate those sites with the content types contained within them. Expect to do some cleanup once the migration process is completed.

Work package. A work package contains a summary of the work assignment and any forms or checklists the user will require while performing the work.

Staffing. This section should describe the staffing model required to execute the public folder migration. Also, skill sets and experience must be listed here.

Training. The document should include training requirements for Help desk staff, IT staff who will support SharePoint, and SharePoint end users.

Testing. A well-defined test plan with scenarios and “How to…” checklists are a must to ensure that the migration occurred as planned. For example, you should perform the following tests:

Risk Management. A common way to address risk is to develop a risk plan document that lists each of the risks. An approach that works well is for you and your team to identify the risks, then analyze and rate each according to probability (high, medium or low) and impact (high, medium or low).

When developing your design document, lean on your Tiger team on a regular basis for design advice, reviews and sanity checks. Ultimately you want to have a weekly discussion with them so that you can stay informed about each others' projects, tasks, and roadblocks. You also want them to be co-authors for the document so they have some skin in the game.

Phase 4: Application Remediation
 This phase deals with the remediation of the applications discovered through the inventory process conducted with the business units during the previous phase. Generally, this is the most time-consuming process; each application is assessed to determine its specific requirements such as technology and level of effort. Generally, applications can be categorized as low, medium or high complexity according to the following guidelines:

• Low-- Your toolset and process has identified a simple solution. For example, simple code changes that remove specific public folder-related APIs and replace them with SharePoint-related APIs.
• Medium--Your toolset and process has identified a solution that involves moderate re-coding and testing. For example, simple code changes that removes specific public folder- and third-party product or Line of Business (LOB) applications-related APIs and replaces them with SharePoint-related APIs.
• High--Your toolset and process cannot identify a solution and therefore more detailed assessment is required. For example, the application requires recoding and additional products such as Office InfoPath to provide forms and SQL Server for a data repository.

I highly recommend that you perform application remediation early in the project. In large organizations, this phase should occur perhaps six to eight months in advance to the data migration phase.

Phase 5: Pre-Migration
This phase is all about making sure you (and your users) are ready to undertake the public folder migration. This phase is mostly about managing quality and risk. Also note that the premigration detailed steps are also specific to the toolset you choose. (For example, reporting is automated or a manual process). The following items must be double checked before migrating:

Phase 6: Migration
This phase consists of the steps in the actual migration of public folder data to SharePoint sites. The actual steps for conducting the migration will depend on the migration toolset you've chosen because screens and options will be different. Therefore, I'll just generalize the basic steps, as follows:

Phase 7: Post Migration
During the post-migration phase, the organization is charged with maintaining the information architecture and enforcing the information management policy. Here are steps your organization can take to facilitate the success of these tasks:

Making the Leap
Microsoft’s investment in public folders has noticeably declined in recent years, and SharePoint is clearly being positioned by Microsoft as the replacement platform. SharePoint offers many comparable features in addition to providing a platform for building, deploying, and managing applications. The decision to migrate your public folders to a platform such as SharePoint is specific to the organization's information strategy and is dependent on factors, such as the complexity of the current deployment, in addition to the availability of the necessary funds and resources.

Related Reading: