Strategies for Migrating Public Folders to SharePoint

Some rumors take on a life of their own even after they’re proven untrue. Microsoft’s support of public folders has fallen into that category since early in the Exchange 2007 development cycle, when it was rumored that public folders would be dropped from the product. The truth was more nuanced: Exchange 2007 supports public folders, so we can continue to use them until the end of the Exchange 2007 product life cycle—2016 at the earliest. However, the writing is on the wall, as Microsoft pointedly didn’t promise support for public folders in future versions. In fact, in a February 2006 blog post (http://msexchangeteam.com/archive/2006/02/20/419994.aspx), Terry Myerson of Microsoft affirmed that public folders “may not be in our next major release after Exchange 2007.” The Exchange team has been recommending that new collaborative applications be developed using the Microsoft .NET Framework and the SharePoint family, but that leaves the problem of moving your existing public folder data and applications from Exchange into SharePoint. Let's look at what public folders do and then talk about a strategy for how to migrate your public folders and their data to SharePoint.

Public Folders in Real Life
Public folders have been a part of Exchange since its initial release; they provide a multiple-master replication model for storing and replicating content between multiple servers. When you take a close look at their functionality, they have only a few basic uses:

Archiving or sharing of email. I use public folders to hold reader mail, and many companies use public folders to capture the contents of Internet mailing lists or other information sources that they want to keep around.

File sharing and Microsoft Office document storage. In this storage role, public folders are used as a place to keep documents so that they’re accessible to multiple people without using a file server. In addition, the documents can be replicated to a server close to the end user, which is often important in large organizations.

Team calendar and contact sharing. In this collaborative role, public folders are used to store contact and calendar data so that it’s available to multiple users.

A platform for applications. These applications are typically based on custom forms that are filled out and stored in public folders; most of the applications now in use were developed some years ago when Microsoft was still stressing this capability.

Over the last two or three years, Microsoft has been positioning the SharePoint family of products as the logical replacement for these public folder capabilities. In this article, I’ll focus on how Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server 2007 (MOSS 2007) support these basic scenarios, although their implementation might be quite different from what you’d do with Exchange. For example, SharePoint supports document libraries, which can be used as a powerful replacement for ordinary file shares and public folders; SharePoint also provides additional attributes, content indexing, and other features missing from Exchange public folders. SharePoint supports check-in/check-out and versioning for document libraries, which is far superior to the conflict detection used for documents stored in public folders. It’s much easier to customize and build applications on top of SharePoint than on Exchange, although this article won’t discuss migrating public folder–based applications to SharePoint.

However, there are two things that Exchange public folders do that SharePoint does not. First, SharePoint doesn’t support data replication, so there’s no way to maintain multiple replicas of your data. Although Exchange public folder replication can be cantankerous at times, it provides a usable way to distribute your data for high availability purposes or to speed up access for remote users. Microsoft recommends you use farms of clustered SharePoint servers, which helps with the availability problem somewhat. However, building a geographically distributed SharePoint farm is still pretty challenging, and it can be expensive because you have to deploy new hardware and software to replace an existing Exchange capability. Second, SharePoint doesn’t provide a built-in method for working offline unless you supplement it with Microsoft Office Outlook 2007 or Microsoft Office Groove 2007, both of which can synchronize SharePoint data to a computer so it’s accessible while disconnected from the network. (For more information about using Groove with SharePoint, see the Windows IT Pro article “Groove 2007,” June 2007, InstantDoc ID 95793.) Outlook 2007 and MOSS 2007 work well together in this mode, so if offline access is important to your organization you’ll have to include Outlook 2007 deployment as part of your overall migration plan. SharePoint also lacks a few additional niceties that we’ve come to expect from public folders, such as the ability to maintain per-user read/unread status.

Choosing a Migration Strategy
The first fundamental decision you have to make is whether or not you’re going to migrate your data. It’s perfectly reasonable to keep your existing public folders active and use them, deploying new applications and workflows on SharePoint. However, as part of their migration, many organizations want to transition to a pure Exchange 2007/Outlook 2007 environment by removing their public folders and using the Exchange availability service for free/busy data and HTTP for Offline Address Book (OAB) downloads.

A further consideration is the timing of your move to SharePoint and the timing of your move to Exchange 2007 and Outlook 2007. Only if you have no pre-2007 versions of Outlook or Exchange can you do away with all your public folder servers. Until that day, though, you need at least one public folder server to hold the system free/busy folder and the OAB distribution folders. For most organizations, that means you can move all of your “real” public folders to SharePoint but you’ll need to keep a single public folder store around for free/busy and OAB, which means you’d better add a second one for redundancy.

I assume that you want to migrate your public folders and their data to SharePoint. To do so, you have to complete six steps:

Preparatory Steps
Before you start on the six steps above, you might want to take some preparatory steps. First, you should probably restrict the ability to create new public folders by using Outlook to adjust the permissions on your existing folder hierarchy so people don't add folders in the midst of your migration. Second, you should start building a SharePoint infrastructure to test. This doesn’t have to be complicated: A simple Windows virtual machine (VM) will provide an adequate sandbox for many needs. This may seem contradictory, as step 4 above calls for you to create a SharePoint infrastructure. The difference is that in step 4, you’re planning an infrastructure to handle your actual production data after its migration, whereas before the migration starts all you care about is the ability to use SharePoint for experimentation. You might even consider setting up a pilot SharePoint environment so that users who need collaboration or team workspace functionality will be able to get it while you build you the production environment. When you do this, remember that you’re going to need SharePoint expertise that you might not have on staff. Don’t forget to budget for this expertise, along with any SharePoint or SQL Server licenses you’ll need.

Surveying Your Existing Data
The first step to an effective migration is identifying what’s in your public folders. There are several ways to accomplish this, depending on how many public folders you have and how important they are. You should begin by using a tool such as PFDAVAdmin or Quest Software’s Quest MessageStats to generate a list of all your folders and their last access and modification times. That will immediately help you identify folders that haven’t been used in a long time. This survey process will take longer if you have many public folders (some organizations have hundreds of thousands). You should keep in mind that SharePoint permissions are different from the MAPI–based permissions used by public folders; the normal practice for setting SharePoint permissions is to assign one or two accounts as the owners of a site, then use Active Directory (AD) groups to grant permissions for readers, contributors, and administrators of the sites and items you want to grant access to.

After you complete the survey process, you have some decisions to make. Exchange public folders mix discussion functionality (which we normally use for email) and document storage. SharePoint uses two separate models for these two different types of data: Document libraries hold documents, and discussion lists hold threaded discussions. If you move Exchange public folder discussions to a document library, you lose the conversation threading from Exchange; if you move documents to a discussion list, you won’t have the versioning and check-in and check-out capabilities.

Spring Cleaning for Public Folders
It’s important to reduce both the number of items and the volume of data in your public folders before you migrate, for several reasons. First, it makes the migration happen faster; second, if you have public folders with lots of items (or very large items), the odds are good that those folders contain old content that you don’t need. Finally, SharePoint libraries aren’t meant to handle more than about 1,000 items per library, so dumping the contents of very large public folders directly into SharePoint will give you poor performance.

Sometimes public folders end up as dumping grounds, primarily because Exchange doesn’t include good tools for cleaning up old or unwanted data. However, you can certainly set size limits on folders to restrict their growth. You can also set age limits on folders so that Exchange will delete items that are older than the age limit. The age limits are based on the modification date, not creation date, of each folder item, so bear in mind that you might see more, or fewer, items disappear than you expect depending on how recently items have been modified. You set both the age and size limits on the Limits tab of the properties of individual public folder databases. The limits you impose apply only to replicas in that particular database; for example, if you set a 30-day age limit in one database and a 60-day limit on another, the database with the 60-day limit will contain items missing from the other database. If you need to, you can override the blanket limits applied to the database by setting individual limits for selected folders; use the Public Folders container in Exchange System Manager (ESM) to find the folder you’re interested in, then open the folder’s Properties dialog box and use the Limits tab to set appropriate limits.

You also need to remove unwanted replicas of your folders. There are two schools of thought about the best way to do this. One school says that it’s best to remove the replicas before you modify any items in the folders; that way you don’t have to use bandwidth to replicate all the changes you make as part of the cleanup. The other holds that it’s a better idea to keep your replicas, but with replication disabled so that changes you make don’t affect the replicas until you’re sure that you wanted the changes.

Choosing a Migration Tool
Microsoft doesn’t ship a tool for migrating public folder data to SharePoint from Exchange. It’s understandable, given that the current version of Exchange still supports public folders perfectly well. Of course, the SharePoint team doesn’t ship such a tool either, which is a little odd given how much it stands to gain from encouraging Exchange administrators to migrate away from public folders.

Such a tool is a necessary prerequisite to carrying out a meaningful migration, as it’s not feasible to migrate more than a few items manually. You can find a variety of free and commercial tools to help you move public folder items from Exchange to SharePoint; each has its own capabilities and limitations. I wrote about several tools in the Exchange UPDATE newsletter (see “SharePoint and Public Folders, Part 2: Migration Options” InstantDoc ID 93127). What should you look for in a tool? Here are a few suggestions.

Reporting mode. A reporting mode is very useful when determining what you want to migrate or when you need to know what would happen if you ran the tool with particular settings.

Flexibility. Some tools give you a way to create rules or XML files that specify where items should be moved from and where they should go. These tools offer more flexibility for large deployments; you should also evaluate whether the tool can be run on a schedule when necessary.

Two-stage migration. Some tools perform direct migrations that move items straight from public folders into SharePoint; others provide a two-stage migration process that copies items from Exchange and stores them in an intermediate format. Each of these approaches has its advantages; I prefer the two-stage process because it gives you more flexibility.

Ongoing synchronization. If you’re migrating public folder content over time, be aware that some tools can provide ongoing synchronization after a migration, while others are intended for one-time use. The likelihood that you’ll need to have ongoing synchronization will increase with the number of folders and contained items that you have.

Customizable. The amount of control you have over the migration process will vary between products. Some are highly customizable, and some are not.

Third-party tools available include products from Quest Software, AvePoint, Tzunami, and CASAHL Technology. There’s also a free alternative, the SharePoint Import and Export toolset available from http://www.spsdev.com/. See “Migrating Public Folders from Exchange to SharePoint,” July 2006, InstantDoc ID 50172, for more details on how to use it. Whichever tool you use, be sure to thoroughly test it before beginning your live migration!

Public Folder and Server Decommissioning
The process for getting rid of public folders is straightforward, but it pays to work through the steps slowly and carefully to ensure that you don’t break any existing clients or applications. Begin by checking each of the mailbox stores in your organization to see what default public folder store they use. Point each database to a public folder database that you aren’t removing.

Next, ensure that any folder you want to keep has at least one replica on a database that you aren’t decommissioning. You can use PFDAVAdmin (which can replace one server with another in a replica list via ESM in Exchange Server 2003 or Exchange 2000 Server, using the Exchange Management Shell in Exchange 2007, or using the public folder scripting interfaces (click the Chapter 9 link at http://www.exchangecookbook.com for samples). If you’re removing all your folder replicas, this step isn’t important, but if you want to keep folders around it’s crucial. After you’ve modified the replica list appropriately, make sure you wait for replication to complete. Depending on your replication topology and the number of changes that need to be replicated, this can take a day or two. Unfortunately, there’s no good way to monitor the replication process other than checking the replication status (which should say “In sync”) and making sure that the items have been replicated by spot-checking them in the target folder.

Finally, you can decommission the database, or server, that you want to remove. Exchange 2003 SP2 won’t let you delete a public folder store until its outbound replication is completed, but earlier versions don’t have that safety check. When you’re ready to remove the last Exchange 2003 or Exchange 2000 public folder server, be sure you follow the steps in the Exchange 2007 documentation (http://technet.microsoft.com/en-us/library/bb288905.aspx) to ensure that you don’t leave any stranded objects. In particular, don’t delete any administrative group that has ever contained mailboxes, because pre-Outlook 2007 clients won’t be able to get free/busy data from the public folder.

A Challenging Process With Benefits
Replacing public folders with a complete SharePoint implementation can be challenging for larger organizations, but the process is fairly straightforward for smaller public folder trees, and the integration of SharePoint with Office, Groove, Live Communication Server, Office Communications Server, and other applications can bring some very valuable benefits. Just make sure to plan your migration thoroughly before you start it.