Managing PST Files in Microsoft Outlook

Update: Since this article was written technology has passed by PST files. It might be time to eradicate PSTs...

When Microsoft Outlook doesn’t depend on Exchange Server for email services, it uses personal folder (PST) files to store mailbox data. Also, where Exchange profiles are in use and Exchange Cached Mode (previously called offline mode) is configured, OST files are stored on the client. An OST file is a slave copy of a specific Exchange mailbox but otherwise maintains the same properties of a PST file. In her article “Common .pst File Questions,” Sue Mosher discussed some of the concerns and solutions associated with managing PST files for users. Most of those concerns remain valid today, but there are some new ones that warrant your attention. To make sound decisions about PST file usage in your environment, you need to understand a little bit about the evolution of PST files in general, as well as the differences between legacy PST files and the newer Unicode-based PST files. You might need to know how to manipulate their location and use according to your unique business requirements.

Personal Folder Evolution

PST files have undergone growing pains over the years. Older PST files had significant limitations that tested users’ patience. Early PST files had limitations of 16,384 folders and 16,384 items per folder. Outlook 98 introduced the Allow upgrade to larger tables setting, which made an irreversible change to the PST file headers and increased the maximum item count per folder to 65,536.

With Outlook 2003 came the option to use a new PST format based in Unicode. Unicode, as the name suggests, provides a single code set in which unique numbers are assigned to any letter or character, regardless of language. This feature provides better support for foreign languages, including Scandinavian languages. The most notable feature of Unicode PST files compared with their ANSI predecessors is the elimination of the 2GB file size limitation, which I’ll discuss later. Figure 1 shows the option to create a legacy ANSI PST file or the new Unicode PST file in Outlook 2003 on Windows XP.

In Outlook 2003, Microsoft made many improvements to PST files, affecting both the ANSI format and the new Unicode PST format. The system now uses multiple indices and larger caches, accessing more content from a cache, including sorting choices. Unicode PST files also benefit from a larger sort buffer, which improves performance and index recoverability. In Outlook 2003 and later, PST files also reduce the amount of file-level fragmentation by increasing PST size in sensible blocks of 2MB each when the PST file is 20MB or larger. (Outlook assumes blocks of 256KB for PST files smaller than 20MB.) This just in time and just enough algorithm makes for more efficient use of drive space. And the system has a better handle on managing table indices to free up resources. For example, Outlook will no longer store indices for smaller tables when they can be easily recreated on demand. PST files created with Outlook 2003 or later recycle item IDs, letting Outlook go beyond the previous limitation of 128 million items over the life of the PST file. However, a PST file can still have a maximum of only 128 million items at one time. Hopefully, you haven’t hit that ceiling!

In addition to offering full Unicode support, Unicode PST files have the benefit of removing the crippling 2GB limitation that legacy PST files face. Perhaps you understand the frustration of hitting the 2GB threshold without reasonable warning and without reasonable or obvious options once you’ve hit it. When trying to access an ANSI PST file that has reached the critical 2GB size, Outlook will return an error message such as Errors have been detected in this file. Quit all mail-enabled applications, then use the Inbox Repair Tool. The Inbox Repair Tool (scanpst.exe) won’t resolve this error. As outlined in the Microsoft article “Oversized PST and OST crop tool” (support.microsoft.com/kb/296088), the data storage file has exceeded the 2GB limit and needs to be cropped. The article provides a link to the Microsoft Download Center, from which you can download a utility called Pst2gb (pst2gb. exe). Pst2gb essentially trims the PST file to a usable size without user input on what content gets unceremoniously removed. The new limit is 33TB—more theoretical than practical. (I don’t think you’ll need to call Microsoft anytime soon for the Pst33tb.exe utility.) And because Unicode PST files also shed the limit of 65,536 items per folder, there’s no limit to the number of items per folder except for the 128 million item limit for the entire PST file, as I mentioned earlier.

Converting to Unicode

The new benefits of Unicode PST files are automatically realized when you create them in Outlook 2007 or Outlook 2003. However, existing PST files in ANSI format represent an administrative challenge should they require upgrading to the new Unicode PST format. You can convert existing legacy PST files to Unicode PST files in several ways.

Although there’s no in-place upgrade for actual PST files, you can use a few methods to move PST content from ANSI PST files to new Unicode PST files manually. First, when you create a new profile without pointing to a specific PST file, Outlook will create a new Unicode PST file in the default location. At this point, you can either use File, Import and select the ANSI PST file or open the ANSI PST file from within Outlook and manually move the data to the new PST file. (Whereas the Import option changes the time stamp on the data to the date and time of the import, manually moving the content between PST files retains the original time stamp.) Second, you can use Outlook’s archive feature to move the data. Create a new Unicode PST file (be selecting File, New, Outlook Data File), then select File, Archive and select the new Unicode PST file as the archive destination. When the data has been archived, select the new Unicode archive PST file as the destination for new content for the profile.

You can also use a third-party application to accomplish this task. For example, Pete Maclean’s Upstart (www.maclean.com/upstart/index.html) can migrate content from a set of PST files in a common location to new Unicode PST files, leaving the originals intact. This solution might make particular sense for enterprise migrations of PST data in which large numbers of PST files need conversion to Unicode PST files. An enterprise solution might save a lot of time from manually migrating large numbers of PST files.

No matter which method you use, you should save a backup copy of the original PST files before moving content. Converting to Unicode PST files isn’t a reversible process, and the result is that Outlook versions prior to Outlook 2003 won’t be able to access content. Trying to use Outlook 2002 or earlier to open Unicode PST files will return an error such as This .pst file is not compatible with this version of the Personal Folders information service. Contact your Administrator.

Are PST Files Bad?

There’s a long-running mantra among Exchange administrators that PST files are “bad.” What those administrators mean is that, for many reasons, a PST file isn’t the best choice as a primary storage location for mailbox data. Consider the following:

Certainly, however, there are times where PST files are beneficial in a corporate environment. Remember that many businesses are using Outlook to access messaging solutions other than Exchange, and these solutions most likely employ Internet protocols POP3 or IMAP4. The latter protocol synchronizes with the server mailbox, but the former might be best configured to leave a copy of messages on the server to reduce allor- nothing dependency on the local PST file. PST files do make inexpensive archives, but from a corporate perspective, archives might need to be centrally managed and searchable for compliance and discovery reasons.

Controlling the Default Folder Location

By default, Outlook keeps PST files in a folder deep inside the users’ profile folder hierarchy, thus providing basic security when other users share the same workstation. In XP, you’ll find the files at Documents and SettingsusernameLocal SettingsApplication DataMicrosoftOutlook, and in Windows Vista, you’ll find them at Users usernameAppDataLocalMicrosoft Outlook.

If you don’t like email storage to be buried in a user profile, or if you use automation across different client OSs, one recommendation might be to standardize the PST file location to a specific place (e.g., D:email). Doing so might simplify an automated backup of workstation email, for example. To configure an Outlook data storage location, you can use the Office 2007 Customization Tool (OCT), which can assign the default folder location for new PST or OST files for Outlook 2007. The tool’s predecessor, the Custom Installation Wizard (CIW) for Outlook 2003, can also perform this function for new installations. Figure 2 shows the OCT option. You’ll find the setting under Features, Modify User Settings. Navigate to Microsoft Office Outlook 2007, Miscellaneous, PST Settings. This configuration won’t move PST files already in use but will assign a default location for their creation.

Alternatively, to change the default path that Outlook uses when creating a new PST file, you can create a registry entry specific to the user. This value works for Outlook 2007, 2003, and 2002. Open your registry editor, and navigate to HKEY_CURRENT_USER SoftwareMicrosoftOfficeversion number Outlook. (The version number for Outlook 2007, 2003, and 2002 and are 12.0, 11.0, and 10.0, respectively.) Add a new string value called ForcePSTPath with the full path as the value. A registry file for this entry for Outlook 2007 would resemble

[HKEY_CURRENT_USERSoftwareMicrosoft  Office12.0Outlook]“ForcePSTPath”=”C:\email”

When creating a new Outlook Data File, Outlook will open this folder location for the user to save his or her new PST file. This location also applies to new OST files.

Configure a Maximum PST and OST File Size

Although Unicode PST files have a theoretical maximum size of 33TB, they still have a default size limitation of 20GB. You can amend this threshold lower or higher through the registry. Certainly, larger PST files aren’t going to perform as well; even 20GB is quite excessive. An administrator or user might want to prevent uncontrolled growth of personal folders on client workstations because of backup and restore resource requirements, prevention of performance degradation, or even disk space concerns. My recommendation would be to limit the size to around 10GB–12GB. To do so, you can configure a user setting at HKEY_CURRENT_ USERSoftwareMicrosoftOfficeversion
numberOutlookPST or a policy setting at HKEY_CURRENT_USERSoftware
Policies MicrosoftOfficeversion numberOutlook PST. Again, the version number applies to the version of Outlook (and Office).

You can create four entries for each of these registry locations. The DWORD values that control this setting for Unicode PST files are MaxLargeFileSize and WarnLargeFile- Size. The latter value doesn’t actually warn the user, but it does let Outlook continue with internal operations while preventing the arrival or creation of additional content. (The PST file can still marginally expand as a result of internal processes when the WarnLarge- FileSize value is reached.)

ANSI PST files have a similar control but with smaller values. Because ANSI PST files still have the 2GB limitation, their registry entry is in bytes and the word Large is removed from the DWORD value names. If the MaxFileSize for an ANSI PST file is set beyond the 2GB limit, the value is ignored.

An example of these registry settings for an Outlook 2007 user with the default values would resemble

[HKEY_CURRENT_USERSoftwareMicrosoft   Office12.0OutlookPST] “MaxLargeFileSize”=dword:00005000“WarnLargeFileSize”=dword:00004c00“MaxFileSize”=dword:7bb04400“WarnFileSize”=dword:74404400

The above DWORD values are in hexadecimal format. You can easily push out these registry entries through Group Policy, the Microsoft Office Resource Kit tools, logon scripts, or another remotemanagement application.

Disable PST/OST Creation

Sometimes in an Exchange environment, you actually want to prevent users from using PST files. Perhaps you have a server-side archive solution, or access to workstations is too insecure for email storage; in such cases, you might require that all content remain in a centralized location. Again, you can accomplish such a configuration through a registry entry that you can then push to systems through Group Policy, logon scripts, the OCT for Outlook 2007, or the CIW for new Outlook 2003 installations. You could also use your registry editor to manually perform the configuration.

Navigate to HKEY_LOCAL_MACHINE SOFTWAREMicrosoftOfficeversion number Outlook, and create DisablePST—a DWORD entry with a decimal value of 1 to toggle it on. This value removes the Outlook Data File option from the File, New menu in Outlook, as well as the File, Archive option. It also prevents the creation of OST files and archive PST files and disallows exporting to or importing from other PST files.

Flexibility and Stability

Newer Unicode PST files bring greater flexibility and stability to managing messaging data with Outlook 2007 and Outlook 2003. Different companies will have different needs regarding email storage, and PST management might be part of that administrative challenge. Remember that you have options available to you for controlling PST file location and size—and even for preventing their use.