PAE and Exchange 2003

Experienced administrators know how much memory Exchange, and the Exchange Information Store process in particular, consumes. Microsoft has improved memory management for Exchange through techniques such as Dynamic Buffer Allocation within the Exchange Server 2003 Information Store. However, Exchange developers still must depend on the underlying hardware and OS to effectively use memory. And since we don't yet have a 64-bit version of Exchange, Exchange is limited to 4GB of address space. However, recent OS and hardware advances help Exchange make the most of 32-bit.

Physical Address Extension (PAE) is a hardware technology that lets Windows applications on IA-32 servers address more than 4GB of physical memory. PAE actually uses 36 bits to create additional addressable memory. PAE lets an OS memory manager use a three-level address-translation scheme to access memory above the 2GB or 3GB available on servers that don't use PAE. For a guide to PAE on Windows, including how the three-level address scheme works, see the PAE X86 Technical Reference at http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/techref/26eccf33-2454-4222-841a-c6d5aa1fc54c.mspx.

Like any mechanism that lets an OS give more memory to applications, PAE reduces the need to write pages from memory to the system page file. Windows Server 2003 and Windows 2000 Server systems can use PAE on IA-32 servers by inserting the /PAE switch in boot.ini. (For more information about setting the /PAE switch, see the Microsoft article at http://support.microsoft.com/?kbid=268363.)

When Intel introduced PAE, Microsoft warned Windows administrators not to enable the /PAE switch because it could cause data corruption in Microsoft SQL Server running on Windows 2003. (For an explanation of the problem, its cause, and the resulting hotfix, see the Microsoft article at http://support.microsoft.com/?kbid=834628.) Out of the box, the Information Store can't access more than 4GB of memory. But because the Exchange development team hadn't tested Exchange 2003 on PAE-enabled systems, Microsoft wouldn't support that configuration.

This situation changed with the release of Windows 2003 Service Pack 1 (SP1), which includes a feature, called data execution prevention (DEP), that requires the PAE kernel. DEP technologies perform additional memory checks to prevent malicious code from running on a system and to prevent code execution from data pages. Both Intel and AMD support DEP in their most recent IA-32 servers, and PAE provides the underlying memory management that DEP needs to run on Windows 2003 SP1 and later and Windows XP SP2 and later. If Windows 2003 SP1 is running on DEP-capable hardware, the OS automatically configures boot.ini to use DEP. (For more information about DEP, see the Microsoft article at http://support.microsoft.com/?kbid=875352.)

In addition, new Intel PCI Express server chipsets require PAE to access all the memory installed on a system. On any system, the chipset uses some memory for I/O resource memory mapping, resulting in a hidden memory space, called memory-mapped I/O, that the OS can't access. In 4GB systems, the memory-mapped I/O space exists outside the 4GB physical space. But if the software supports only 32-bit physical addresses, it can't access more than 4GB of memory, resulting in some memory that applications can't access if they run on hardware equipped with older chipsets.

So how does all this affect Exchange? Because it's a database at heart, the Exchange Information Store uses memory cache for as much of its data as possible. After Windows 2003 SP1 and the new chipsets came along, the Exchange development team tested Exchange 2003 SP1 and SP2 on Windows 2003 SP1 servers with PAE and DEP enabled and found no problems. Accordingly, Microsoft now supports these configurations. You can also enable PAE for Exchange 2003 on Win2K Server systems as long as you've applied Win2K SP3 or later and the hotfix at http://support.microsoft.com/?kbid=834628. Microsoft supports this configuration but doesn't recommend using it unless you need to access all the memory on servers equipped with newer PCI Express chipsets.

Even if you aren't using PAE, you can control memory use on Exchange 2003 servers running on Windows 2003 systems that have more than 1GB of physical memory. Microsoft recommends using the /3GB and /USERVA=3030 values for all mailbox and public folder servers when you configure boot.ini switches. Also, if your system has more than 1GB of physical memory, the Exchange Server Best Practices Analyzer (ExBPA) will check the HeapDeCommitFreeBlockThreshold registry setting on Exchange 2003 and Exchange 2000 Server systems to ensure that the system is optimally tuned for proper Information Store operation. (For more information about this functionality, see the Microsoft article at http://www.microsoft.com/technet/prodtechnol/exchange/exbpa/cbc2b55c-f850-47f7-a414-cc01b91a01a6.mspx.)

The Information Store benefits the most from correct configuration of memory settings, so you don't need to set the switches on front-end or messaging bridgehead servers; they don't use the Information Store much. However, bridgehead servers that handle traffic through third-party Message Transfer Agent (MTA)-based connectors might benefit from memory tuning because Exchange channels third-party application traffic through the Information Store. Microsoft has updated the most recent version of ExBPA—version 2.1a when I wrote this article—to check for correct boot.ini switches on running servers. You can download ExBPA at http://www.microsoft.com/exchange/downloads/2003/exbpa/default.mspx.

One caveat: Microsoft recommends against setting the /3GB switch on systems that act as Windows domain controllers (DCs) or Global Catalog (GC) servers or on systems running Microsoft Small Business Server (SBS). For more information about tuning boot.ini settings for Exchange, see the Microsoft article at http://support.microsoft.com/?kbid=823440.