Geoclustering

I've recently seen a surge of interest in an exotic technology. I'm not talking about High-Definition TV (HDTV)--I'm talking about geoclustering: building clusters of servers that are in physically separate locations. For example, a company in a metropolitan area such as Detroit could run Exchange Server on a cluster, placing one cluster server at headquarters in Dearborn, Michigan, and another at a factory in Melvindale, Michigan. The company could connect both servers to a storage unit at a facility somewhere else in the area.

Although geoclustering can be prohibitively expensive, Exchange does support the technology, which seems to be drawing more and more attention. The appeal of geoclustering from a disaster-recovery or business-continuance standpoint is obvious. By physically separating cluster nodes, you gain a high degree of protection from events such as fires, building collapses, and earthquakes. How does geoclustering work? Well, the bottom line is optic fiber. Simply put, geoclustering extends the length of the optic fiber run used to build a Fibre Channel Storage Area Network (SAN). Of course, a lot of rather complicated technology is involved. For example, long fiber runs require amplifiers; most geoclustering implementations use multiple channels (e.g., 32 channels of 32MBps each) which in turn require switching and routing equipment.

You can implement geoclustering in one of two ways. You can split the cluster so that each node is in a separate location and all nodes share a common storage system in yet another location. In this case, the limiting factor is the latency that occurs while packets traverse the fiber between each node and the storage unit. And of course, this design is vulnerable to storage failures. The more sophisticated (and expensive) design uses local Fibre Channel SAN storage at each cluster node; the SAN hardware synchronously replicates this storage to the counterpart nodes. The advantage of this method is that major SAN vendors understand synchronous SAN replication and have a lot of experience designing and deploying replicated systems. If you have problems with replication, your vendor will be the first line of support.

As interesting as it is, geoclustering isn't for everyone and has significant limitations. First, Exchange doesn't recognize the physical design of the underlying cluster. If the cluster works (e.g., if it has acceptable latency and I/O policies), Exchange will work. If the cluster doesn't work or works only intermittently, the same will be true of Exchange.

Second, geoclustering is incredibly expensive. I've seen geocluster implementations that cost several million dollars. All that fiber (and its support infrastructure) costs money, and the vendor-specific replication solutions you need to use are expensive as well. The additional latency caused by the length of the fiber run means that Exchange can't handle as many concurrent I/O operations as it can on a traditional cluster or individual servers, so the number of users you can support on a geocluster shrinks as the fiber run length increases. In effect, you have to spend much more to support the same population of users.

Third, geoclustering requires extraordinary caution in design. The I/O profile for Exchange is very demanding, and an improperly designed cluster will provide an endless source of frustrating, difficult-to-fix problems caused by I/O latency. The Windows Server Catalog lists several Windows Hardware Quality Labs (WHQL)-certified geocluster implementations (under Cluster Solutions, Geographically Dispersed Cluster Solution); choose one of these if you want to deploy a ready-made geocluster rather than build your own. You should also work with your SAN vendor to make sure that it will stand behind its recommended design as suitable for (and supported with) Exchange.

The bottom line is that Exchange supports geoclustering but that few applications justify its present-day cost or limitations. Still the technology is very cool--almost as cool as HDTV--and might be looking forward to a bright future.