Fewer Web Servers, More Uptime

Control is Jason Cornellier's watchword. Jason, a senior-system engineer at Ford Motor Company, oversees IT operations for a Web-server farm that serves Ford's corporate locations worldwide. Since Jason and his IT team upgraded the Web servers to Microsoft Internet Information Services (IIS) 6.0 about 3 years ago, thousands of Ford Web users have seen a significant increase in server uptime—with downtime occurring rarely—mainly, Jason says, because of IIS 6's granular resource-usage monitoring and application isolation that let IT keep a close eye on Web-farm operations. Senior editor Anne Grubb spoke with Jason about how IIS 6 has helped Ford consolidate servers, improve server troubleshooting and availability for end users, and gain tighter control of Web applications' usage of system resources.

As I understand, you oversee an IIS 6-based Web farm that provides development, quality assurance (QA), and production hosting for internal Ford Motor Company users. How many servers are in the farm, and what applications are they hosting?

We run a shared farm that serves more than 125 sites, for all divisions of the business—basically every employee internal to Ford Motor Company. The user base is well over 100,000, excluding external users. The sites range from extremely complex sites—those running Active Server Pages (ASP), COM/COM+, and Windows .NET Framework applications—to a few that serve straight HTML pages. They run all types of applications, depending on the line of business. Content on separate servers is rolled through development to QA to production status by an automated workflow process.

In the farm in general, we run a secure and highly available infrastructure with an extremely high level of disaster recovery. In this way, we can host applications from [those having] the lowest service level agreement (SLA) to the highest in a single farm. We also use proactive Network Load Balancing (NLB) and content failure detection and have a cross-site and intersite failover disaster recovery plan. Finally, we run all the servers with a NAS back end, which centralizes all our storage, so that we don't have to duplicate content on every single server.

Talk a little more about your high-availability strategy.

Basically the farm—the development, QA, and production environment—is split 50-50 between two different sites. Each site has the equivalent hardware, storage, and so on, and we use some high-end networking technologies to provide active failover and detection. We use this setup not only for disaster recovery and high availability but also for general maintenance. For instance, when we have to apply patches, we just pull [the servers that need to be patched] out of the load-balancing mix, apply the patches, and move the servers back into the farm. So we don't see any downtime when we do patches or upgrades.

How did moving from IIS 5 to IIS 6 improve your Web operations? Did end users see any benefits from the upgrade?

Mainly, moving to IIS 6 has improved uptime; our uptime is up to close to 100 percent on all the sites now—even the most unstable sites. IIS 5 didn't offer quite the level of application isolation that we can get today in IIS 6.

Another benefit is that we can get many more sites on fewer boxes. So we're getting a much higher level of consolidation. The original Web farm, which ran on IIS 5, had more than 50 servers. Going to IIS 6 has created a lot of efficiencies in our hardware environment and allowed us to eliminate more than half those original servers. We're now spending less than half the cost of the original farm for the existing farm. In fact, the hardware that we're on right now is up for lease shortly, so we'll be replacing it. Because of IIS 6, we might be able to scale down even further, although we're adding more sites and features.

Is that cost savings primarily on hardware or in other areas, such as IT operations?

It's in both areas—hardware savings as well as because of the IIS 6 efficiencies that improve troubleshooting for our IS staff. IIS 6 provides a high level of resource-usage monitoring and resource allocation, which lets us pinpoint sites that are in their trouble zone as well as create per-site, proactive automatic support rules. If a site is having issues, we can limit them to certain percentages of CPU and memory. This prevents a site from exceeding [its allocation] so that it can't affect other sites [by consuming more than its allotted resources]. And as well, with IIS 6's Worker Process Isolation Mode, we can identify all the sites very quickly and see, for example, whether their COM objects might be breaking the site.

What tools and procedures do you use to secure the Web farm?

Applications are run by using individual credentials with no specific user context behind them. We have a process to change those passwords frequently, and since each ID runs under its own security context, it's much easier to map applications' permissions on the back-end NAS—for example, what data they have access to—so that they can't share data, nor can they access each other's data, although they all reside on the same server. [This security scheme] also makes troubleshooting easier because it lets us identify a site that a process is running on just by looking at the ID. Additionally, we run a Secure Sockets Layer (SSL) environment, as do most Web sites.

What are your future plans for the Web farm and IIS?

IIS is the core of our Microsoft shared environment. By that, I mean that, for our environment, we feel confident in the uptime availability of what we created here, so that instead of buying individual servers for individual products—for example, dedicating a server to Microsoft SharePoint Portal Server—we're going to integrate [the applications] into this farm and keep building on it as our Microsoft platform.

So you're saying that IIS will be the basis for meeting all your Web-oriented and collaboration needs?

IIS is the basis for our Microsoft shared application environment, and we're piloting new Microsoft collaboration tools. There's no reason that this farm isn't more than capable of handling those needs. In the past the Microsoft perspective has been "one server, one application." But we've followed a consolidated strategy long before it was popular, and it's really paid off.