7 Important Microsoft Exchange Server Innovations

The benefit of hindsight is a wonderful gift to technologists. It lets us understand how changes that were made in the past laid the foundation forcurrent capabilities. In the case of Microsoft Exchange Server, three types of deployment are now possible: classic on-premises, cloud with MicrosoftOffice 365, and hybrid, in which some parts of the infrastructure reside on premises and some reside in the cloud.

Exchange didn't arrive at this point by accident. The Exchange development group has done a lot of heavy lifting over the past decade so that customerscan enjoy the choices they have today. Looking back, I see seven important areas in which innovation has liberated Exchange from its origins as aLAN-based email server running on Windows NT. Apart from all else, I think that these areas represent the most crucial areas for Exchangeadministrators to master in the foreseeable future.

The Magnificent Seven

Perhaps it's unfair to focus on just seven areas, when Exchange is now a huge product that spans well over 20 million lines of code. I freely admitthat other areas of innovation within Exchange deserve consideration.

For example, there's the elegance of single page patching within a database availability group (DAG). This implementation allows the Information Storeto detect page corruptions and then broadcast requests to other database copies to retrieve the data necessary to fix the problem, all while keepingthe database online and serving users.

Even so, I'm happy to stay with the group that I've selected. In no particular order of importance, these are my chosen areas of innovation:

1. Remote Procedure Call (RPC) over HTTP

2. Windows PowerShell

3. Autodiscover service

4. Mailbox Replication Service (MRS)

5. Multibrowser interfaces

6. Storage improvements

7. Client Access server

Now, let me explain my logic.

#1: RPC over HTTP: Eliminating VPNs

In the early days of Exchange, remote access was characterized by synchronization woes, slow dial-up connections, and VPNs. Improvements in clienttechnology -- such as the drizzle-mode synchronization that Microsoft Office Outlook 2003 introduced -- and the now-ubiquitous nature of fast wirelessconnections have eliminated the first two issues. And the need to establish a VPN before connecting to Exchange was firmly whacked on the head by theintroduction of RPC over HTTP in Exchange Server 2003.

Now known as Outlook Anywhere, RPC over HTTP was initially difficult to configure. Those who persisted and published the necessary public connectivitypoints discovered that Outlook could connect easily. Furthermore, it could use the public Internet to transport, safely encapsulated in HTTP packets,the Messaging API (MAPI) remote procedure calls (RPCs) that form the basis of any communication between Outlook and Exchange. Administrators loved thefact that ports 80 and 443 were the only ones that they needed to open in the firewall, especially because these ports are usually open to supportother web-based applications.

Since Exchange 2003, Microsoft has gradually improved the configuration and management of this component. Now, Outlook Anywhere is a real strength ofthe product, so much so that it provides the fundamental underpinning of both Microsoft Business Productivity Online Standard Suite (BPOS) and Office365.

After all, requiring every customer to create a VPN to access Microsoft Exchange Online would be impossible and too expensive to create and maintainfor many small-to-midsized businesses (SMBs). Such a requirement would also create a huge burden for Microsoft, which would need to manage the incomingVPN connections.

Simply put, Outlook Anywhere allows everyone to connect across the Internet. Plus, the $6 per month price point for Office 365 is feasible. That's whyRPC over HTTP is #1 on my list.

#2: PowerShell: Delivering a Common Management Platform

PowerShell might seem a strange choice for #2. But its introduction in Exchange Server 2007 and subsequent upgrade to remote PowerShell in ExchangeServer 2010 have delivered many benefits. First, PowerShell provides consistency across the management interfaces within Exchange. In other words, youcan use the Exchange Control Panel (ECP) to update a mailbox's properties in Office 365, or you can run the Set-Mailbox cmdlet by using PowerShell.Both routes lead to the execution of the same logic.

But my main reason for selecting this component is that the advent of remote PowerShell delivers the ability to manage Exchange Online without needingto log on to the servers on which Exchange runs. Obviously, Microsoft couldn't permit thousands of Office 365 customers access to mailbox or HubTransport servers. But remote PowerShell allows domain administrators to connect across the Internet and validate their credentials for a tailoredsession that contains the exact set of cmdlets that they're allowed to run. And because PowerShell forces all paths to the same logic, the user canconnect by running Exchange Management Shell (EMS), or through ECP, or through the Microsoft Management Console (MMC)-based Exchange Management Console(EMC): Everything works in the same way. That's why PowerShell is my #2 choice.

#3: Autodiscover: Solving User Pain

Microsoft introduced the Autodiscover service in Exchange 2007 as a solution for the perennial problem in which users had trouble configuring Outlookwith the parameters that were necessary to connect to Exchange. The basic difficulty: Server names that make perfect sense to administrators put usersinto a deep sleep. Apparently, regular users have problems coping with names such as EXSERVER1 or MBXHUB-LONDON.

Microsoft figured out that the issue could be solved by making computers communicate to figure out a user's mailbox location. That's what Autodiscoverdoes, by consulting signposts such as a service connection point (SCP) in Active Directory (AD) to retrieve information about Exchange and to discovera mailbox's current location. Of course, AD is available only inside a firewall, but Autodiscover can use URLs that are published to the publicInternet to retrieve what it needs. This capability laid the foundation to allow easy connection to Exchange Online in BPOS and Office 365. WithoutAutodiscover, administrators would face far more complexity and cost when they configured user PCs to connect to the cloud.

Autodiscover works well for Outlook 2010 and Outlook 2007, the only two PC clients that support the feature. It also works well with the latest ActiveSync clients, Outlook 2011 for Macintosh, and Entourage 2008. Microsoft expanded the information thatAutodiscover returns to the client in Exchange 2010. Autodiscover can now retrieve data about alternative mailboxes that Outlook should open (i.e., mailboxes to whichthe user has been granted full access) and URLs that map Exchange services such as the Offline Address Book (OAB) distribution point, Exchange WebServices, Unified Messaging, and so on. Exchange returns all this information as XML data to Outlook, which refreshes the information every 15 minutesto ensure that it keeps up-to-date with any changes.

You can use Outlook's Test E-mail AutoConfiguration option to gain an insight into the information that Autodiscover retrieves. To run the option,press CTRL and right-click the Outlook icon in the system tray and select Test E-mail AutoConfiguration from the menu. Clear the Use Guessmart andSecure Guessmart Authentication options, enter your email address and your password, and click Test. Outlook goes through the same process that it useswhen it populates a user profile for the first time. Click the XML tab to see the data returned by Exchange.

As Figure 1 shows, this data includes details such as the server name and the URLs that are necessary to connect to Exchange Web Services (EwsUrl) andECP (EcpUrl). In this example, I've connected to my Office 365 mailbox, so the results are those that come back from Office 365.

Figure 1: Results of an Autodiscover test run 

It's worth noting that Autodiscover is one reason why Microsoft doesn't support connecting Outlook 2003 clients to Office 365. Although you couldmanually configure all the server settings in Outlook 2003 to make a connection to a cloud-based mailbox, the automatic nature of profile updates thatAutodiscover performs in Outlook 2010 and Outlook 2007 facilitates easy movement of mailboxes between servers. Microsoft continually adds servers as itbuilds out its Office 365 infrastructure, and then rebalances workload across available servers by transferring mailboxes between databases. If Outlook2003 clients were connected, users would need to know what to do if their mailboxes were moved. The result might be a support nightmare. Restrictingsupport to Outlook 2010 and Outlook 2007 solves that potential support headache and encourages users to upgrade to software that can take advantage ofthe most recent server features (e.g., MailTips), so it's a logical win-win situation for Microsoft.

Autodiscover keeps Office 365 administrators from running around to help users more than they already need to do, so it's a good choice for #3.

#4: Mailbox Replication Service: Smooth, Elegant Moves

Large mailboxes are all the rage today. I can't quite understand why, as the vast majority of humans take a long time to fill a 25GB mailbox. Still,the advent of massive mailboxes has made the task of moving them a much more complex business than it used to be.

Until Exchange 2010, administrators had to ask users to log off before their mailboxes could be moved. The mailboxes moved in real time, and nothingcould be done until the full and complete mailbox reached its destination. If something happened during the move, you had to restart. Althoughworkable, this solution is ineffective when mailboxes need to move from on-premises servers to the cloud and vice versa. And as I mentioned earlier,Microsoft needs to operate in a state of almost perpetual mailbox moves to rebalance databases as it builds out its Office 365 infrastructure.

Cue the introduction of MRS in Exchange 2010. Mailbox moves now occur in the background, with frequent checkpoints and sufficient intelligence toacknowledge when the receiving server might be under pressure. Although moves cannot be scheduled (a deficiency that Microsoft will surely address inthe future), they can be auto-suspended. Such moves copy an initial snapshot of a mailbox's contents, in the background, and then pause. Anadministrator can check the mailbox move report and resume the move if all seems well. MRS then takes another snapshot of the source mailbox andperforms an incremental synchronization before switching pointers in AD to direct the user to the new location.

Best of all, MRS is extremely competent at transferring mailboxes between AD forests, from on-premises deployments to the cloud, and between versionsof Exchange (from Exchange 2003 onward). Without the service it's difficult to see how Microsoft could cope with the transfer of millions of mailboxesto Office 365; the manual workload involved in setting up and managing mailbox moves across the Internet would be massive and costly. Instead, manyorganizations use the auto-suspend technique to move mailboxes in the background, as many as 30 days before a user is finally transferred to Office365.

And MRS does more than move mailboxes. Microsoft has expanded its responsibilities to manage tasks such as import and export of mailbox data frompersonal folder stores (PSTs) and mailbox restore requests. Because of its key role in mailbox management, I think MRS deserves its position as #4 onmy list.

#5: Multibrowser Support

No self-respecting cloud service can run without offering browser support. In Exchange, browser connectivity began in 1997, when Exchange 5.0 shippedthe first version of Outlook Web Access (OWA), now charmingly renamed Outlook Web App (but still OWA). In truth, the first version of OWA wasn't verygood. But much has been learned about browser interfaces, standards, and functions since 1997.

Today's OWA boasts a solid and highly functional UI that puts competitors such as Google Gmail to shame. OWA connects with ease to both on-premises andcloud versions of Exchange and supports a rich user experience on Internet Explorer (IE), Google Chrome, Mozilla Firefox, and Apple Safari. Otherbrowsers can connect with the basic version of OWA -- still highly usable, even if missing some of the bells and whistles of its premium counterpart.

An equally important evolution is occurring for browser-based management interfaces. Exchange 2010 introduced ECP -- and promptly confusedadministrators because some tasks, such as discovery searches or ActiveSync device policy maintenance, can be performed only through ECP. No trace ofthese tasks appears in the flagship EMC. Equally frustrating, some tasks, such as DAG management, show up only in EMC. For example, DAG management canbe performed only through EMC, which boasts other convenient features such as wizard-based execution of complex tasks (e.g., certificate management)and the ability to show administrators the underlying PowerShell code that will be executed to accomplish a task.

However, the important thing to realize is that ECP is in its first iteration. Web-based management is a crucial piece of the evolution of ExchangeOnline within Office 365. Over time, I expect more and more administrative tasks to show up in ECP, until the need to maintain two management consolesdisappears. I predict a time when Microsoft removes EMC from Exchange, leaving us with a single browser-management client that hopefully includes allthe EMC features that administrators depend on.

ECP already supports the same range of browsers as OWA, and Exchange 2010 Service Pack 1 (SP1) introduced the ability for users who aren't mail-enabledto use ECP to perform administrative tasks. Because of their key roles for both users and administrators (today and in future cloud services), I givethe OWA/ECP combination the #5 slot on my list.

#6: Storage: Cheap and Cheerful Bytes

Storage was the single biggest cost bucket for Exchange 2003 deployments. This can be partially explained by the cost per gigabyte back in 2003 to2005, but the I/O profile that Exchange generated was more relevant. To be blunt: Exchange 2003 was an I/O hog. The only way to get reasonableperformance was to deploy expensive storage that could deliver the required I/O operations per second (IOPS) performance to support the desired numberof mailboxes. SAN storage -- expensive to buy and equally expensive to set up and maintain for Exchange -- was often necessary.

Microsoft tweaked the Information Store in Exchange 2007 to improve its I/O profile, with considerable success. Fundamental change then occurred inExchange 2010, with the first database schema change since Exchange Server 4.0. Many other updates were made to drive down I/O requirements and deliverthe ability to run Exchange on cheap DAS and Just a Bunch of Disks (JBOD) arrays. As a result, the I/O requirement of one IOPS per mailbox that existedin Exchange 2003 is now down to circa 0.1 IOPS. Furthermore, Exchange 2010 can process huge mailboxes stuffed with hundreds of thousands of items -- asituation that would have brought Exchange 2003 to its knees.

If Microsoft hadn't invested in the engineering effort to make such a dramatic improvement to database performance, it wouldn't be able to offer 25GBmailboxes in all but the most basic Office 365 kiosk plans, and it would be unable to compete with Gmail. For those reasons, storage improvements areinnovation #6.

#7: Client Access Server: Exchange's Black Box

The Client Access server role first appeared in Exchange 2007, as the common endpoint for Internet protocols. The role was then enhanced with the RPCClient Access layer in Exchange 2010, to become the endpoint for MAPI clients. The Exchange 2010 Client Access server also hosts MRS, so it plays animportant role in mailbox moves.

The transition from mailbox server to Client Access server is why I think this component is important. Without the Client Access server, Exchange wouldhave been unable to break the connection between mailbox and server that existed prior to Exchange 2010. The high availability gained through DAGswould also have been impossible because we would be unable to switch databases easily and quickly between servers. And without DAGs, Microsoft probablywould have been unable to deploy Exchange Online as it has, with multiple database copies split across geographically separate data centers. The upshotis that Microsoft might not be able to offer the financially backed 99.9 percent service level agreement (SLA) that it offers for Office 365.

But there's more. The Client Access server is key to the effective management of incoming client connections. On-premises administrators will be awareof the interaction between firewalls, load balancers, and the Client Access server to manage HTTP, MAPI, IMAP4, POP3, and ActiveSync clients. BetweenWindows Server 2008 R2 and Exchange 2010, important changes -- such as the ability to combine individual Client Access servers into Client Accessserver arrays and better handling of session identifiers -- allow servers to handle a greater volume of connections more effectively. This ability ishugely important, given the tens of millions of clients that Office 365 is expected to manage. That's why the Client Access server takes the finalposition in my list.

I'd be happier with my choice if Microsoft provided functions to allow administrators to understand exactly what the Client Access server is doing atany point. Right now, the Client Access server is often a black box, with connections going in and out with no indication of whether everything isprogressing as expected. Perhaps Microsoft will do something to improve the telemetry and feedback from the Client Access server in a future version ofExchange, if only to help the folks who run Office 365.

What's Next?

I won't be offended if you disagree with the logic behind my choice of the magnificent seven. Have fun reordering my list to assign your best idea ofthe relative importance of each component. The important thing is that Exchange has been gradually evolving over the years, to the point that it cannow cope with the demands of three very different kinds of deployment.

I'm not sure that the engineers realized how all these pieces would fit together when they set out to design any one component, but blessed serendipityhas brought us to the current situation. I'm quite sure that development isn't complete and that you'll see massive evolution in Exchange over the nextfew years, as the product heads for its 20th anniversary in 2016. If we revisit the assessment at that time, I think some of the same components willbe on the list. The question is which new components will appear over the next four to five years?