Exchange Server Performance Standards

It’s time to discuss performance, again. I don't often write about this topic because, based on feedback from organizations running Exchange Server, performance hasn't been much of a concern since Exchange Server 5.5 shipped in 1997. However, with Exchange 2000 Server in the pipeline, this is a good time to discuss performance and scalability from an Exchange 2000 perspective.

In the world of Exchange performance, the Messaging API (MAPI) Messaging Benchmark (MMB) has become the defacto standard that vendors use to demonstrate Exchange Server performance. Although MMB has its weak points, the benchmark is a fair evaluator of a standardized workload on a particular hardware platform. However, MMB is not a realistic representation of a typical corporate workload, especially for disk I/O. In most cases, MMB fails to exercise disk subsystems to the point of real-world workloads. I've been involved with Exchange Server performance testing since 1996, and I've seen cases in which the medium MAPI user profile that MMB uses does mimic real-world environments; however, in most cases, MMB workload representation is too light.

For Exchange 2000, the performance and scalability focus will shift beyond MAPI workloads to Internet protocols such as POP3, IMAP, and HTTP. Although Exchange Server 5.0 and 5.5 provide support for Internet protocols, large-scale deployments are rare. I expect Exchange 2000 to be a real contender as an Internet protocol-based messaging and collaboration server. For ISPs such as Data Return and United Messaging, who are hosting messaging services using Exchange Server, performance and scalability of both MAPI and Internet protocols is of paramount importance. However, no clear benchmarking standard exists for Internet messaging protocols.

Corporate customers, however, will still be concerned primarily with MAPI in the short term. Enter MMB2. MMB2 is a new MAPI benchmarking standard that Compaq and Microsoft are developing for Exchange 2000. To find out more about MMB2, read my column in Exchange Server UPDATE, December 10, 1999. Go here and click Previous Issues, December 10, 1999.

As Exchange 2000 gets closer to shipping, many organizations are wondering about the performance results. Microsoft’s goal for Exchange 2000 was to offer MAPI performance equivalent to Exchange 5.5. Thus far, based on the limited test data I've seen, Exchange 2000 appears to be able to deliver on those promises. However, because Exchange 2000 Release Candidate 1 (RC1) is the latest release available and not all of the performance work is complete, we have to wait for the final results. If you plan to conduct performance characterization for Exchange 2000 using RC1, I suggest that you focus on understanding how these exercises will be conducted for Exchange 2000 rather than on getting actual results. When the final releases of Exchange 2000 and MMB2 are available, you can hit the ground running.