Exchange Server 2013 Transitions from RPC to HTTP

Once you hace deployed Microsoft Exchange Server 2013 SP1 (or later), one of the more interesting features you can take advantage of is MAPI over HTTP (code-named alchemy). MAPI over HTTP provides the ability for Messaging API (MAPI) clients and servers to communicate across a HTTP connection without using remote procedure calls (RPCs). Ever since its debut back in 1996, Exchange and its clients have used RPCs. The elimination of the now-aged mechanism marks the conclusion of a modernization process that began more than a decade ago.

The Roots of RPC in Exchange

RPC has a long and noble history. In the early 1990s, Microsoft built on the work of the Open Software Foundation (OSF) to create RPC as an interprocess communications (IPC) mechanism that underpinned client-server applications such as Outlook and Exchange. Applications that use RPCs don't have to worry about the details of communication across local and remote networks through different protocols because the RPC layer is responsible for this activity.

Related: Exchange Server 2013 SP1: A Mixture of New and Completed Features

In effect, RPCs allow applications to get on with the task of providing their unique functionality instead of having to constantly reinvent the networking wheel, even if occasionally you need to mess around with settings to make everything work. (The method described in "XGEN: Changing the RPC Binding Order" might bring back memories—or nightmares—about some of the fine-tuning required in early Exchange deployments.) In early Exchange deployments, the RPCs connecting Exchange and its clients traveled across many different protocols, including TCP/IP, NetBIOS, and named pipes. Over time, the focus shifted to TCP/IP, which became the de facto standard for Exchange 2000 and later.

The Problems with RPC

Although RPC delivers significant advantages to application developers, it's an old mechanism that was originally designed to work across LANs rather than across the Internet. The age and relative lack of recent development in the RPC mechanism is reflected in much of its documentation, such as the "How RPC Works" article, which is based on Windows Server 2003 and last updated in March 2003.

Today, more and more of our communications flow over the Internet. The trend to use cloud-based services is just one influence that has driven traffic to the Internet. Mobility is another important influence, as people increasingly take advantage of sophisticated mobile devices and high-speed wireless networks at work and at home. Both influences cause problems for applications that use RPCs.

The first problem is that RPC communications are sensitive to disruptions due to network hiccups (a well-known feature of the Internet). RPCs use fixed buffer sizes, which means that an application like Outlook might have to make multiple calls to a server to retrieve or send information. This situation isn't improved by the fact that MAPI is a verbose protocol, where the transmission of messages from client to server involves a mass of properties and values.

Outlook does its best to insulate users from network glitches. The introduction of the cached Exchange mode, drizzle mode synchronization, and various optimizations to minimize network consumption in Outlook 2003 provided users with a reliable and robust solution for working when networks weren't dependable. At that time, the problem was more with dial-up telephone connections than Wi-Fi, but the foundation was set and Outlook has built on it ever since. It's doubtful that Microsoft could've been so successful with Office 365 if Outlook didn't work so well across the Internet.

However, things are a bit uglier behind the scenes. If network glitches occur, Outlook can end up in a cycle of constant retries to perform common operations such as downloading new items. And the sensitive nature of RPCs mean that Outlook often has to restart activities because a disruption happens, such as moving between two wireless access points. The chatty nature of RPCs, the increasing size of messages, and large attachments (e.g., digital photos, music files, Microsoft PowerPoint presentations) all make for more extended connections. The extended connections, in turn, expose those communications to more disruption. In the end, Outlook consumes many bytes on the wire just to get email updates done.

Microsoft introduced RPC over HTTP connections—aka Outlook Anywhere—to help with this situation. It was first used by Exchange Server 2003 to allow Outlook 2003 to connect to mailboxes without creating a VPN. However, it's also a fairly old technique that hasn't really changed much since its introduction in Windows 2000. Plus, Outlook Anywhere poses some unique challenges of its own. For example, HTTP is a half-duplex connection—in other words, it can carry traffic in a single direction. RPCs need full-duplex connections with synchronized inbound (RPC_IN_DATA) and outbound (RPC_OUT_DATA) links. Outlook Anywhere solves the problem by using two HTTP connections to carry the RPC traffic and session affinity to keep the links synchronized with each other. This arrangement is well known to administrators who set up load balancers to deal with Outlook Anywhere connections. Microsoft eased the problem somewhat in Exchange 2013 by moving the responsibility for handling session affinity to the Client Access Server, but the two connections are still used.

Microsoft has a lot of experience with other clients that use HTTP without RPCs. Both Exchange ActiveSync (EAS) and Outlook Web App (OWA) clients use longstanding HTTP connections to communicate with Exchange. These connections don't require the complicated handshaking used when RPC connections are made, which means that they work much better than Outlook over low-quality links. Mobile devices in particular tend to hop between networks all the time, which creates a challenge in terms of maintaining connectivity with a server. EAS and OWA are both able to manage this kind of environment better than Outlook, which continually loses and restores connections at the expense of a great deal of network activity, most of which is hidden from end users by the cached Exchange mode.

Similar to EAS and OWA, MAPI over HTTP uses a kind of "hanging GET" transaction for Outlook notifications. Simply put, the client establishes a long-lived HTTP GET to the server so that the server knows the client is interested in notifications such as the arrival of new mail. If something happens on the server, the transaction is closed and the client issues a command to fetch the new data. If nothing happens during the GET interval (e.g., 20 minutes), the transaction is closed and a new HTTP GET is established.

Outlook is a more complicated and sophisticated client than an OWA or EAS client (e.g., Mobile Outlook on Windows Phone). As such, you'd expect that Outlook consumes more bandwidth and executes more transactions with Exchange and other servers. For instance, Outlook 2013 fetches social media information from Facebook and LinkedIn, whereas OWA and EAS don't. More important, when Outlook synchronizes with a mailbox, it downloads full-fidelity copies of items and attachments and usually processes every folder in the mailbox. Some control over the amount of data downloaded is available by restricting Outlook 2013 to downloading a certain window of data. However, Outlook typically performs a complete synchronization, whereas OWA or EAS typically concentrate on just a few folders and perhaps just the last few weeks of information from those folders. In addition, OWA and EAS don't download attachments until requested by the user.

All this is of little importance when Outlook connects using a high-quality, fast network. It becomes very important when the network experiences frequent drops and reconnects, or when the bandwidth proves insufficient. At this point, an OWA or EAS client becomes a lot more usable in terms of getting work done. I discovered this in a very real sense when travelling in Australia last year, where some of the public Wi-Fi networks are less than good. OWA or EAS worked fine. As for Outlook, let's just say that I used OWA far more than usual during that trip.

The complexity of troubleshooting Outlook connections with Exchange is also cited as a reason for change. Once multiple layers are involved, it becomes more difficult to trace connections and determine the root cause. Problems with Outlook connecting to Exchange (both on-premises and to Office 365) have been at the top of the support issues list for years, and the developers hope that simplifying the communications layer will make support easier. Tools such as the Microsoft Exchange RPC Extractor are available to help parse network captures and interpret the contents of RPCs, but the output can be difficult to understand (The TechNet Magazine article "How IT Works: Troubleshooting RPC Errors" provides some useful background.) The hope is that it will be easier to make sense of the HTTP traffic in conjunction with other data (e.g., IIS logs) to understand the flow of client connections.

So, in a nutshell, RPC is:

This sounds like an opportunity for improvement, which is exactly what the MAPI over HTTP initiative seeks to deliver.

MAPI over HTTP

Although not much has happened in the RPC world for the last decade, the same isn't true for HTTP. It has received a huge amount of development attention from leaders in the industry, including Microsoft. Apart from bug fixes, not much can be expected for RPC in the future, so if you were a development group, would you put your proverbial eggs in the HTTP or RPC basket? Aside from the inevitable problems involved in making a change to the way products like Outlook and Exchange work, the choice seems pretty obvious if you want to take advantage of new techniques and features that are likely to come along with the HTTP protocol over the next few years.

Figure 1: Understanding the Two Modes of Communication (Source: Microsoft)

Figure 1 shows the two modes of communication that Outlook 2013 SP1 can use with Exchange 2013 SP1. On the left, you have the older RPC-style connections over either TCP/IP (for Exchange Server 2010 and earlier) or HTTP Secure (HTTPS—for Exchange 2013). In this instance, RPCs form the middle ground to link the client and server. However, that middle ground creates an extra layer of complexity if a need arises to debug connections. On the right, you can see how MAPI over HTTP replaces the dependency on RPC by directing the remote operations executed by Outlook across HTTP connections. This is different than RPC over HTTP because the RPCs used to carry MAPI instructions (wrapped in TCP/IP or HTTP packets) are no longer used. Instead, the MAPI instructions are sent directly over an HTTP link, which is what most Internet applications use to convey their data.

A change like this involves a great deal of work on both the client and server, which is the reason why it only works when a supported client (Outlook 2013 SP1 or the upgrade released on December 12, 2014 for Outlook 2010) and Exchange 2013 SP1 work together. Remember that RPC is designed to relieve applications from the need to worry about IPC by providing a library of common functions that the applications can use to make connections. For backward compatibility with older clients and servers, Outlook 2013 SP1 and Exchange 2013 SP1 still contain all the RPC code, but they also have a new communications library that can be used to direct connections across an HTTP link.

When Outlook 2013 SP1 connects to an Exchange 2013 SP1 server, it advertises its ability to use MAPI over HTTP by sending an indicator in its initial connection. If MAPI over HTTP is enabled for the organization, Exchange responds by providing a set of URLs in the Autodiscover XML manifest returned to Outlook. The URLs point to the connection points (an IIS virtual directory for MAPI over HTTP) that Outlook can then use to establish the HTTP connection. If Outlook connects to Office 365, it receives only a set of external connection URLs, whereas on-premises Exchange servers (in a hybrid or pure on-premises configuration) transmit both internal and external connection points. Outlook attempts to use the internal connection first, then fails over to the external connection if necessary. Each set contains URLs for the mail store (mailbox databases) and directory, corresponding to the emsmdb and nspi interfaces used by Outlook to access mailboxes and the address book. (The Exchange Team Blog "Understanding how Outlook, CDO, MAPI, and Providers work together" provides some useful information about these interfaces.)

Exchange 2013 supports Outlook 2007 clients too. Although Outlook 2010 has now been updated to support MAPI over HTTP, given the age of the client, the update will not be back ported for Outlook 2007, so they'll continue to use RPC over HTTP. As future clients and servers are delivered over time, Microsoft will remove support for RPC over HTTP in the same way it removed support for RPC over TCP/IP (in Exchange 2013) and UDP (initially in Exchange 2010 and finally in Exchange 2013).

Bumps Along the Road

It all sounds simple, but like many other changes in technology, using MAPI over HTTP requires some planning, testing, and other work on the part of administrators. First, MAPI over HTTP isn't enabled by default. You have to enable it by updating the organization configuration to let mailbox servers know that it's OK to tell Outlook clients that MAPI over HTTP is available.

Figure 2: Enabling MAPI over HTTP

As shown in Figure 2, this is a one-time operation performed by running the Windows PowerShell command:

Set-OrganizationConfig -MapiHttpEnabled $True

This shouldn't be done until the entire organization is upgraded to Exchange 2013 SP1 and you have deployed Outlook 2013 SP1 clients. Office 365 tenants won't get to vote on when this update happens, as the responsibility for making the change lies in the hands of Microsoft.

Second, switching network communication mechanisms doesn't come without some pain for clients. Outlook maintains information about its configuration in MAPI profiles. The change to MAPI over HTTP requires that the profiles be refreshed with the new information. Afterward, Outlook has to switch to the new MAPI over HTTP endpoints. The switch can't be done on the fly, so Outlook has to exit and restart. Users will therefore see the infamous message: The Microsoft Exchange administrator has made a change on the server that requires you to quit and restart Outlook. After Outlook is restarted, it connects to MAPI over HTTP and everything starts to flow as planned.

A side effect of the switchover to MAPI over HTTP is that the Outlook profile page that controls the Outlook Anywhere settings, which Figure 3 shows, will no longer be visible. This is logical because Outlook Anywhere is no longer in use, but you might have to update end-user documentation to reflect the new reality.

Figure 3: Configuring Outlook Anywhere Settings Will No Longer Be Required

If all the servers in the organization run Exchange 2013 SP1 (or later), the change to MAPI profiles should be a one-off affair. However, if you maintain some down-level servers (earlier versions of Exchange 2013, Exchange 2010, or Exchange 2007) and move mailboxes from Exchange 2013 SP1 to those servers, the MAPI profiles for those mailboxes will need a further refresh.

Users might or might not be concerned about having to restart Outlook. If users accept this kind of thing, they'll restart Outlook as directed. However, if users are worried about it, they might create a Help desk support ticket. Given that the switchover is a one-time organization-wide event, you might have a situation in which hundreds of users return to work after a blissful weekend, resume their PCs from hibernation, see the message, and call the Help desk. Some careful communication and guidance to users is required here.

Third, no real information is available yet about the possible impact on performance (client or server) or network bandwidth. Although RPCs can be fragile, they're compact. Pure HTTP traffic is likely to generate more bytes on the wire. For instance, the Exchange developers have decided to use HTTP headers to carry diagnostic information in MAPI over HTTP connections. This information is useful in debugging connectivity problems that might arise, but the additional traffic might be a problem for companies that depend on remote connections, such as those with highly mobile users or Office 365 tenants. Microsoft is likely to release more data about bandwidth requirements that compares MAPI over HTTP with RPC over HTTP. This data (or your own data) should be used to determine whether your network connections' capacity needs to be upgraded. In addition, if you use appliances (e.g., load balancers, WAN accelerators), you should check with the vendors to determine whether any configuration changes should be made to optimize for MAPI over HTTP traffic.

Finally, there's always the potential for edge cases that aren't tested during development to pop up soon after an update is shipped to customers. Thinking about what might happen here, you need to check if you have any particular aspect of your environment that Microsoft is unlikely to test. For example, you might have a particular Outlook add-on that might cause problems after the switchover. Given that add-ons depend on Outlook networking, I think that this is an unlikely scenario, but it's wise to test the entire configuration of your operational environment as thoroughly as possible before enabling such a profound change.

Microsoft believes that the transition to MAPI over HTTP will help Outlook improve its ability to work in a world where mobile communications are an absolute requirement. It makes sense for technology to adapt and embrace new conditions rather than look back to a time when the only available networks were dial-up telephones and safe corporate Ethernets. RPCs have had their day. They'll remain in use for as long as old clients exist (and are supported), but the writing is firmly on the wall. HTTP-based connections are the lingua franca of the Internet and that's what Outlook will use in future.

The Right Approach

Installing Exchange 2013 SP1 across an organization won't force you to switch to MAPI over HTTP. For now, you can continue to use RPC over HTTP until you are ready to switch, most likely sometime in the future when you've upgraded all your desktops to Outlook 2013 SP1 or applied the hotfix for Outlook 2010. It's possible that there will be a time when using MAPI over HTTP is mandatory, but that's likely to be well in the future when new versions of Exchange and Outlook are built to use only this mechanism.

I see lots of good things about this transition. It doesn't seem to make a lot of sense to keep old networking and communication technologies in place when they struggle to deal with modern operating conditions. Developing a new approach that's designed to cope well with the kind of connections being used today seems like a good idea. Although you can expect some bumps along the road until the inevitable configuration, operational, and programming problems are solved, I hope that the implementation occurs without undue disruption to end users. When it comes to technical evolution, good planning, comprehensive testing, and solid execution seem like the right approach to take.