Migrating Public Folders from Exchange to SharePoint

Every time Microsoft releases a new version of Exchange Server, we wonder whether it will still support public folders. Over the years, companies have experienced wild growth in public-folder usage. Users are often creative with public folders (which are intended for data archive), using them to make backups of their desktops and even to run business-critical applications. However, with its recent announcement of Exchange Server 2007—the next major version of Exchange—Microsoft has set the grounds for "de-emphasizing" public folders.

Terry Myerson, the general manager of the Microsoft Exchange Server Group, explains Microsoft's public-folder strategy for Exchange 2007 in his blog (http://blogs.technet.com/exchange/archive/2006/ 02/20/419994.aspx). To answer questions about the future of public folders, he makes several key points. In new Exchange deployments, public folders will be optional with the use of Microsoft Office Outlook 2007 and won't be installed by default. For companies migrating existing deployments, public folders will operate with little change from Exchange Server 2003. (Upgrades to Exchange 2007 won't depend on migrating or decommissioning public folders.)

These points have left companies wondering, "If we don't use public folders, what will replace them, and how do we get there?" Terry suggests in his blog that companies looking to replace Exchange public folders should consider Windows SharePoint Services (SharePoint). SharePoint, which is included with Windows Server 2003, is a Web-based service that lets users perform team collaboration and document sharing. What kind of process are you looking at if you decide to move from public-folder functionality to SharePoint? Let's take a look at SharePoint and public folders from a strategic standpoint, then discuss the process of embarking on a data migration.

SharePoint vs. Public Folders
In his blog, Terry notes that Microsoft will fully support Exchange 2007 for 10 years. With that in mind, what are the advantages of using SharePoint? SharePoint is a robust collaboration solution that offers some enticing advantages over public folders. Public folders are great for archiving data, but they lack features such as document management, automatic notification of content changes, integration with Microsoft Office Live Communications Server and InfoPath, and integrated content search.

SharePoint is essentially a .NET Web application that uses Microsoft SQL Server as its data store. SharePoint boasts the concepts of lists and libraries, each of which offers unique properties and capabilities. Examples of lists include discussions, contacts, tasks, and events. Lists aren't document-based. There are three library types: documents, forms, and picture libraries. In document libraries, you can keep a collection of documents or other files that you want to share. Document libraries support features such as subfolders, file versioning, and check-in and check-out. With this basic understanding of how SharePoint stores items, you'll see how having separate list and library types can pose a problem for data migration.

In contrast with SharePoint, public folders combine the functionality of SharePoint's discussion lists and document libraries as a single storage type. Before you begin your migration, you'll have to make a choice about where to move your public folders, and this choice will have an impact on the available feature set. For example, in one public folder, you might have a mixture of Office documents (called SmartDocs) and threaded discussions. If you move all the contents to a document library in SharePoint, you'll lose the conversation association (or threads). If you move all the items to a discussion list, you'll lose the document-management capabilities. It's probably better to make a choice and keep the folder in a particular list type. Splitting the list might cause confusion, and users won't know where to find the old information they once found in a familiar place.

Another major difference between the two products is security models. Exchange provides many granular roles, so users can tune permissions on a folder to delete only their own content. SharePoint doesn't provide item-level security; it provides security only at the list and library levels. So, you can either create one-off libraries with custom permissions or live with the new security model in which anyone can possibly delete anybody else's documents. If a public folder has many access controls applied to it, migration of that folder will result in either multiple SharePoint libraries or a relaxing of that folder's security. These kinds of differences between the products show how a migration will be more complicated than you think.

The Migration Process
For the migration project in this article, I'll use a couple tools from Microsoft's GotDotNet.com site. At this site, you'll find extractor programs that essentially take data from a certain source, such as Exchange's public folders or the Windows file system, and convert that data to an intermediary format that a SharePoint importer program can use.

You'll find the installation packages and source code for the Exchange Public Folder Extractor and the SharePoint Products and Technologies Import Wizard at http://www.gotdotnet.com/workspaces/ workspace.aspx?id=6996fb17-2a54-4607-983b-35c7697baa53. The source code is freely available, so the tools are constantly evolving and getting better thanks to community support and interaction. (You can also consider a few commercial products, such as Tzunami Deployer and CASAHL's migration solution, both of which offer benefits such as full customer support.)

The migration process consists of three steps—cleaning up your public folder environment, extracting content from Exchange public folders, and importing that content into SharePoint—and the use of three separate tools: a cleanup tool called PFDAVAdmin (available for download from Microsoft at http://www.microsoft.com/downloads/details.aspx?familyid=635be792-d8ad-49e3-ada4-e2422c0ab424&displaylang=en) and the aforementioned extractor and importer tools. After the cleanup process, you'll extract the Exchange public folder content to a shared file path in a common export format. Then, from that shared file path, you'll import the information into SharePoint. I'll take a look at each step in depth. Figure 1 shows a public folder tree that I'll use to illustrate a sample migration.

Cleanup
To have the best chance at success, you need to clean up your public folder environment before moving any data. Cleanup and reduction in content—for example, consolidation of folders and eliminating aging content—is important for several reasons: First, it's unlikely that this data will be cleaned up once it's on the new system; second, less content means fewer bytes on the network during migration; and third, fewer tools are available for cleaning up the data once it's in SharePoint. One of PFDAVAdmin's features is its ability to create a content report, which displays the number of items in, size of, and most recent modification of any folder. Using this information can help you identify old content or frequently used folders.

If a folder contains a large number of items, it probably isn't being used for collaboration. Consider moving the data to a file share or near-line archival storage. One reason to not use a document library for data storage is that SharePoint's document libraries don't perform effective enumeration beyond a thousand entries per folder. Therefore, if you migrate a public folder containing thousands of items to a single SharePoint list, performance latency will occur when someone navigates to that SharePoint list.

Next, identify folders that applications are using as data stores, for workflows, or for custom forms. No tools will identify such folders, but messaging administrators are likely already aware if any exist. When you identify an application folder, consider using alternative data stores— for example, you can use a database or leave the existing application in place until the environment is fully on Exchange 2007 and Office 2007. (SharePoint has completely different access APIs and object models, so there's no easy way to port existing code or forms.) These folders aren't good candidates for migration to SharePoint, and in general they're probably not a good long-term location for public folders, either.

Finally, to cut down on old content, consider turning on limits and aging, if you aren't already doing so. (See the Microsoft article "HOW TO: Configure Storage Limits on Public Folders in Exchange 2003" at http://support. microsoft.com/?kbid=823144.) You can set limits as a default for all folders, and you can override the limits on a per-folder basis. After you've cleaned up as much as possible and have identified critical folders, you're ready to begin the migration.

Exchange Extraction
Now, you're ready to extract content from Exchange public folders. You can install the Exchange Public Folder Extractor on any client machine that has .NET Framework 1.1 and Outlook configured for Collaboration Data Objects (CDO) 1.2.1. The account running the Extractor must have Read permissions to the Exchange public folders, as well as Write permissions to the output directory. For the purpose of this example, I'll demonstrate the interactive GUI, but you can also run the Extractor from the command line.

After you launch the Extractor, you'll need to fill in all the fields of the Exchange Public Folder Export Wizard's Exchange Settings tab, which Figure 2 shows. You select a folder from which to start the export, and you can either process all its child folders or select the Export only this folder check box to process only that folder. The Add Webpart check box specifies whether the target list will have a Web Part created on the SharePoint home page. If you don't select this check box, the SharePoint list will be available only on the SharePoint site's Quick Launch bar.

After you enter all the necessary information, go to the wizard's Message Settings tab, on which you can filter the source content by attachment extension or by date. Because a public folder that contains Mail and Post Items doesn't have an exact equivalent in SharePoint—unlike the other types of public folders (i.e., calendar equals events, and contacts equals contacts)—you must choose the discussion type. The tool needs to know which type of SharePoint list or library to convert these folders to. As I discussed earlier, SharePoint has document libraries and discussion lists, each with unique features. Discussion lists maintain message threads, and the message text is migrated as the post's text. Any attachments are linked to the post. Subfolders of the parent become new discussion-board lists. If you select mail type instead of discussion type, you lose message threading (i.e., the ability to group messages based on the same conversation). Subfolders are migrated as subfolders in the document list (similar to a file system view, such as Windows Explorer). In the example that Figure 3 shows, you can see how the Quick Launch bar appears in SharePoint depending on the target library type. Mail is selected on the left, and Discussion is selected on the right. The Mail conversion keeps the parent/child relationship between the folders, as it existed in Exchange, but with the Discussion type selected, discussion lists don't support sublists and the folders are migrated into separate, peer-level lists.

The wizard's third tab is SharePoint Settings. Here, you set the name of the SharePoint target site, which can be an existing site or a new site. This tab is also where you set the contact account and email address, which the wizard sets as the site owner property on the target SharePoint site. On the final tab, Export, you can initiate the migration and view the export progress.

SharePoint Importer
After the export is complete, you're ready to finish the migration with the SharePoint Products and Technologies Import Wizard. Unlike the Extractor, the Import Wizard must be installed on a SharePoint Web server. The account that runs the Import Wizard needs permissions to update the registry on the server that it's running on. The account also must have permissions to create sites and areas on the SharePoint site. The SharePoint Products and Technologies Import Wizard requires .NET Framework 1.1.

On the Import Wizard's Settings tab, which Figure 4 shows, enter the location of the exported files in the Source directory field, and enter the URL to the SharePoint server in the Target field. You needn't include the site name in the URL because it was already specified during the extraction and is stored in the struct.xml file, which is in the folder you specified during the public-folder extraction. You have a few other options for altering import behavior—for example, you can allow importing into existing sites or libraries. On the Import tab, you initiate the import process and monitor the progress as it runs.

Limited Integration
Moving content from one platform to another isn't as simple as merely moving data. You have to consider the differences between the platforms, such as the underlying storage mechanisms and security models) that make some features difficult or impossible to migrate.

A primary difference between SharePoint and Outlook is that SharePoint is a Web-based application; users access their content through a Web browser. With Outlook 2007, we'll likely see tighter integration between Outlook and SharePoint, so users will be able to access more information through the Outlook client. Today, you'll find limited integration between the two products. For example, you can access SharePoint event lists (i.e., calendar) in Microsoft Office Outlook 2003, but the information is read-only. To learn more about Office 2007, check out Microsoft's Office 2007 preview site (http://www.microsoft.com/office/preview).

In terms of collaboration capabilities, SharePoint is a terrific alternative to public folders, but it probably isn't a 100 percent fit for all the data that exists in your public folder system today. Users will enjoy the many benefits of SharePoint, which offers a true collaboration solution in which Microsoft continues to make large investments in the future.