Win2K Datacenter Server

Microsoft aims for the MAJOR LEAGUES with this designated heavy hitter

Microsoft designed Windows 2000 Datacenter Server, the most powerful OS offering in the Win2K family, to meet the needs of customers that require large, mainframelike systems possessing superior availability and outstanding scalability. Until now, Win2K and its predecessor, Windows NT 4.0, couldn't compete with large UNIX implementations. With Datacenter, Microsoft aims to level the playing field by exploiting the largest, most powerful systems ever built for Windows.

One way to describe Datacenter is that it's Win2K Advanced Server plus Service Pack 1 (SP1) plus extra features, available only from OEMs on tested and approved hardware, and offered with optional joint support from the OEMs and Microsoft. The extra features improve on Win2K's scalability, availability, and manageability. And special certification and support requirements in Datacenter further differentiate this OS from the rest of the Win2K server family. Table 1, page 50, compares Datacenter with Win2K AS and Win2K Server.

32 CPUs
With Datacenter, Microsoft has made important strides in advancing Win2K's scalability. Datacenter includes all the scalability tricks in Win2K Server and Win2K AS, plus some new things that you might not have seen yet.

Datacenter marks several "firsts" for Microsoft, one of which is the ability to support up to 32 CPUs in one machine. This capability is the linchpin in Microsoft's scale-up strategy for the Win2K server family.

With NT Server 4.0, Enterprise Edition (NTS/E), you can license one system for as many as eight SMP processors, although systems with four to six processors provide the best performance for the price. Microsoft has dramatically improved processor scalability throughout the Win2K server family, and this improvement is especially apparent in Datacenter. Systems that support 32 processors, such as Unisys' ES7000, are becoming more readily available, so customers can begin to realize the benefits of Win2K's CPU scaling capabilities.

Microsoft made several optimizations in the heart of Win2K to achieve better SMP scaling. These optimizations consist of increased parallelism combined with fewer serializations, and improvements to core functions such as I/O performance, device drivers, and TCP/IP stack performance.

Win2K has more per-processor granularity for system pools and lists than NT 4.0. Each processor has paged and nonpaged look-aside lists for memory allocation, thread pools, and I/O completion ports. Other scaling improvements include more use of fibers (i.e., lightweight threads) to reduce memory and context-switching costs for applications that use fibers, and reduced use of Page Frame Number locks, which was achieved by enlarging the cache virtual address by 50 percent and by implementing a new least recently used (LRU) algorithm.

Microsoft designed Win2K for SMP from the kernel out. The benefits include more linear SMP scalability (i.e., performance improves as you add more CPUs; it doesn't flatten out after four to six CPUs as in NT 4.0), better CPU affinity capabilities (you can associate some processes with a specific CPU—this can help performance by reducing the overhead of context switching as Win2K moves a process from one CPU to another), and better performance for the price when you add CPUs. The four-to-six processor "sweet spot" of NT Server 4.0 is definitely a thing of the past.

Enterprise Memory Architecture
Datacenter supports as much as 64GB of physical memory; Win2K AS supports as much as 8GB. Win2K (like NT) is a 32-bit OS, which means that processes have access to a flat 4GB (2³²) of address space. So how do Datacenter and Win2K AS benefit from more memory? The Win2K Enterprise Memory Architecture (EMA) provides two ways to exploit the larger amounts of memory that the Win2K server family supports: Microsoft's 4GB RAM Tuning (4GT) and Intel's Physical Address Extension (PAE). Applications that take advantage of EMA technologies will scale better than those that don't. Depending on how applications use EMA, they might or might not require modifications to take advantage of the technology.

4GB RAM Tuning. With Microsoft's 4GT (which has been around since NTS/E), the OS typically allocates each process 4GB of virtual memory: 2GB for the application and 2GB for the system. Because the processes don't use all 2GB of their system address space, Microsoft created 4GT to increase the application virtual memory from 2GB to 3GB and reduce the system virtual memory space from 2GB to 1GB, without adding any new APIs. Applications written to take advantage of this extra memory, such as Microsoft SQL Server, can exploit 4GT for better performance.

To enable the 4GT feature when you start Datacenter, you must append /3GB to the Advanced RISC Computing (ARC) path in your system's boot.ini file:

multi(0)disk(0)rdisk(0)partition(1)WIN2K="Microsoft Windows 2000 Datacenter Server"/basevideo /3GB

To use 4GT with an application, you must set the IMAGE_FILE_LARGE_ADDRESS_AWARE bit in the application's executable file header. You can set the bit with the linker switch / LARGEADDRESSAWARE or with the Imagecfg utility as follows:

imagecfg ­l .exe

For more information about using 4GT, see the Microsoft article "Information on Application Use of 4GT RAM Tuning" (http://support.microsoft.com/support/kb/articles/q171/7/93.asp).

In Win2K, 4GT is available only in Win2K AS and Datacenter. Datacenter automatically ignores memory above 16GB when 4GT is enabled because systems using memory above 16GB need 2GB of virtual address space to contain all the necessary page table entries. The administrator who enables 4GT is also choosing not to use memory above 16GB, even if it's present in the system.

PAE. Intel's PAE, the other EMA technology, is new with Win2K and provides access to as much as 64GB of memory in Datacenter (8GB in Win2K AS). In the past, 32-bit Intel processors could address only 4GB of memory. But Intel has extended the addressing capability of its PAE-enabled processors to 64GB (36 bits). PAE requires an x86 Pentium Pro processor or better, more than 4GB of system RAM, and a 450NX-compatible or later chipset. Check with your vendor to ensure your hardware is compatible and tested with PAE.

Microsoft modified Win2K to support PAE natively in the kernel, so you might expect that you don't need to modify the OS or applications to take advantage of PAE's benefits. This expectation is correct up to a point. A Win2K AS or Datacenter system supports PAE memory above 4GB up to the level of the OS with no application modifications—if you want to run multiple applications that require no more than 4GB of memory each. The left side of Figure 1 shows this approach to using PAE.

In this scenario, each application runs unmodified in its normal 4GB of virtual address space (with 2GB for the application and 2GB for the system). Datacenter acts as a consolidation platform, stacking up more simultaneously running applications than ever before possible in Win2K. In addition, this PAE approach greatly reduces paging operations because more memory is available for system cache. You don't need to modify the applications because the Win2K kernel manages the physical location in memory of each application's 4GB of virtual address space. However, you do need to add the /PAE parameter to the ARC path in boot.ini as follows:

multi(0)disk(0)rdisk(0)partition(1)WIN2K="Microsoft Windows 2000 Datacenter Server"/PAE /basevideo

A second way of employing PAE lets you modify an application to use more than 4GB of memory. Address Windowing Extensions (AWE) are a small set of new APIs in Win2K that developers can use to leverage large amounts of memory in their applications. You allocate a memory "window" inside an application's 4GB process virtual address space, along with an amount of physical memory, and the application can access memory through the process virtual address space window as needed. Theoretically, your application could allocate all the memory on the system (up to about 62GB on a Datacenter system). Performance isn't a problem because the processor hardware manages the memory window. The OS doesn't spend unnecessary cycles on overhead associated with mapping memory into the window. AWE-enabled applications can support larger in-memory data structures, larger caches, and larger in-memory databases—all of which extend Win2K scalability and performance. The right side of Figure 1 illustrates this approach.

Windows Sockets Direct
Windows Sockets Direct (WSD) supplies a way for System Area Networks (SANs) to bypass the relatively slow IP stack, thereby providing Winsock applications direct access to SAN hardware for high-speed data transfers. Thus, Datacenter enables great scalability for distributed and parallel applications using SANs. WSD is available only in Datacenter.

A SAN is a distinct class of network architecture that uses a high-performance interconnection between secure servers. This "network within a network" delivers extremely high-bandwidth data transfers (greater than 1Gbps) in a reliable, low-overhead, low-latency manner. SANs use switches to route data; a hub typically supports from four to eight or more nodes. You can cascade hubs to create larger networks. Cable-length limitations range from a few meters to a few kilometers.

The problem with SANs is that their transports, although highly reliable, are proprietary because the network interfaces are implemented directly in the SAN hardware. However, most Windows applications use TCP/IP and Winsock. Thus, without WSD, a Windows application vendor that wants its software to work with a SAN must add a translation layer between TCP/IP and the SAN's proprietary transport. As Figure 2 shows, Datacenter WSD supplies a simulated TCP provider (i.e., the Winsock switch) layered over a TCP/IP provider and a SAN provider to give unmodified Winsock applications access to SANs.

In the WSD stack, the Winsock switch decides whether the network traffic goes through the regular TCP/IP stack or bypasses the TCP/IP stack and goes to the native SAN Winsock provider. Further, WSD can perform direct I/O on the SAN hardware from a regular Datacenter user-mode process, letting regular programs benefit from using the SAN, if the SAN hardware supports direct I/O. The bottom line is improved performance versus regular TCP/IP, as Figure 3 shows.

Applications can realize measurable performance benefits when they use WSD rather than the TCP/IP stack to access a SAN. In addition, applications don't require modifications to work with WSD, and they need not understand proprietary SAN transport interfaces. For more information about WSD and SANs, see the Microsoft articles "Description of System Area Networks" (http://support.microsoft.com/support/kb/articles/q260/1/76.asp) and "Differences Between a System Area Network and a Storage Area Network" (http://support.microsoft.com/support/kb/articles/q264/1/35.asp).

Cluster Service
Availability is paramount in an enterprise-class OS. In addition to possessing measurably better stability than NT 4.0, Datacenter provides significant enhancements to the availability features of the Win2K server family.

Cluster service, known as Microsoft Cluster Server (MSCS) in NT Server 4.0, exists primarily to provide increased availability through application failover and failback and during rolling upgrades. Failover happens when an application fails on one cluster node and another cluster node picks up the job. Failback happens when the original node resumes operation after the failed application is restored. Cluster service manages the failover and failback of applications running on a cluster, without losing any data associated with the failed application. A rolling upgrade lets you upgrade an application one cluster node at a time so that you don't have to take the application down during the upgrade process.

Suppose you need to keep a SQL Server database running. With Datacenter, you could configure a 4-node cluster, as Figure 4, page 54, shows. After you install the Cluster service software and the cluster-aware version of SQL Server, you can configure the cluster so that SQL Server will fail over to an available node. Because users will undoubtedly have open sessions on the server when it fails, Cluster service must perform the failover quickly and automatically, without losing anyone's data. The available node picks up the workload and the data from the failed node. This action is called maintaining state. Cluster service performs stateful clustering because it maintains the user and application state across a failover. The two additional nodes in Datacenter provide an extra level of availability not present in 2-node Cluster service clusters.

Network Load Balancing
Win2K's Network Load Balancing (NLB) service, known as Windows NT Load Balancing Service (WLBS) in NT Server 4.0 and NTS/E, is essentially an IP load balancer. NLB distributes the incoming load of IP requests across multiple nodes running the NLB software to provide scalability and availability. The nodes appear to the outside world as one IP address. As user demand for networked server resources grows, you simply add more nodes to the NLB cluster.

Figure 5, page 54, shows an NLB front end to a business-critical server such as SQL Server or Microsoft Exchange 2000 Server running on a Cluster service cluster. The NLB front end offloads much of the communication burden that the SQL Server or Exchange 2000 cluster would otherwise incur. Along with the load balancing comes increased availability because the NLB cluster routes client requests to the server. If one NLB node goes down, the others will seamlessly pick up the extra load, and the user will notice no interruption in service.

The underlying NLB software is a network device interface specification (NDIS) driver that sits between the NIC and TCP/IP. You install the driver on each server in an NLB cluster. All NLB nodes—or servers—share a virtual IP (VIP) address (registered with DNS) that represents the desired networked resource. All NLB servers listen to all user requests, but only one responds. A load-balancing scheme based on a fast-hashing algorithm that incorporates the client IP address, its port number, or both determines which server responds. You can specify an affinity to allow varying amounts of traffic between servers (i.e., you can specify that some servers should get more traffic than others). A heartbeat feature quickly lets all the NLB nodes know about any changes in the cluster, such as a failure or the addition of a node. When changes occur, NLB starts a convergence process that automatically reconciles the changes in the cluster and transparently redistributes the incoming load.

In contrast to Cluster service, NLB is a stateless clustering solution, meaning that it doesn't retain user and application state across a failover. You typically use a stateless cluster such as NLB to distribute load across multiple Web servers. However, NLB has provisions for maintaining user state, which is necessary for some types of Web server applications, such as online purchasing. NLB is available in Datacenter and in Win2K AS.

For more information about Win2K clustering, see "Microsoft Clustering Solutions," November 2000. For more information about load balancing, see Tao Zhou, "Web Server Load Balancers," April 2000.

Manageability Features
Absent in Windows versions earlier than Win2K is a way to group processes so that the OS can treat them as a unit. A Win2K job object is such a feature—it's a group of processes, usually related, that you can secure and manage as a unit. Datacenter offers two ways to access job objects: Process Control, which is a Microsoft Management Console (MMC) snap-in, and the Proccon command-line utility. With the process control utilities, you can control job-object resource allocation, creation, maintenance, and removal. Furthermore, both Process Control and Proccon will work on any Win2K system, including Win2K Professional, to remotely manage jobs on a Datacenter system.

Alternatively, you can use Windows Script Host (WSH) with standard programming languages to access a job object programmatically. The job-object API set is straightforward and is available across the Win2K server family, not just on Datacenter. If you like to program, I encourage you to try the APIs. The Win2K Platform software development kit (SDK) includes details about how to access job objects. Figure 6 lists the job-object attributes that you can manipulate.

Job objects have many practical applications. You can limit resource usage to control runaway applications. You can set processor affinity to scale applications to the appropriate number of CPUs. And you can use job objects to help meet and maintain service level agreements (SLAs). Changes you make to job objects are persistent, so they will endure through reboots and application restarts, and you can make changes on the fly (i.e., no delay or application restart is required).

Suppose you have an application that has five processes running on a Datacenter machine. You want to limit the working set of memory that the processes can use. The only way to limit the working set is to create a job object that contains all the processes or a job object for each process. If the working set is exceeded, an event will be posted to the event logs. Finding and manipulating job object properties is simple when you use the Process Control snap-in, as Figure 7 shows.

Datacenter, along with Win2K AS, also offers enhancements that ease management and administration of a Cluster service cluster. These include an easier installation and setup process for Cluster service, support for rolling upgrade of applications on Cluster service nodes, the ability to use the Active Directory (AD) class store to centrally manage Cluster Resource DLLs, network and disk Plug and Play (PnP) support, better MMC integration, and a COM automation interface to the cluster API. In addition, Datacenter adds Cluster service support for Win2K infrastructure components such as Microsoft Dfs, Network News Transfer Protocol (NNTP), SMTP, DHCP, and WINS to existing Cluster service support for file sharing, print spooling, Microsoft Message Queue Services (MSMQ), Microsoft Distributed Transaction Coordinator (MS DTC), SQL Server, Exchange 2000, Microsoft IIS, and generic applications and services.

Improved Support
One of the features that most distinguishes Datacenter from other Win2K server versions isn't really a feature at all. Rather, it's a process that Microsoft hopes will make Datacenter support easy and seamless for customers. In an ideal world, customers can call one number to resolve a problem when something goes wrong with their computer. They don't need to endure vendor finger-pointing, and they aren't passed from one vendor to another. They have one point of contact that is available 24 * 7, understands their problem, and can quickly fix their computer.

Microsoft's first stab at this ideal support world is the Datacenter Joint Support Queue. Because customers purchase Datacenter from an OEM rather than directly from Microsoft, they will contact the OEM for support. Typically, one or more Datacenter service professionals from Microsoft will be at the OEM support site. When a call comes in, the folks staffing the joint queue are trained to resolve the problem quickly and efficiently. If they determine that a third-party independent software vendor (ISV) or independent hardware vendor (IHV) product is part of the problem, they will contact the vendor on behalf of the customer. If the product is certified to run with Datacenter, the vendor must have a 24 * 7 support center that the joint queue can contact directly. Ideally, customers never know what happens behind the scenes—they just get a return call from their OEM with a fix for their problem.

If this scenario sounds too good to be true, it probably is. Some variables exist that might make some customers' support experience better than others. For starters, OEM joint queue support is optional; customers must contract and pay for this support. Furthermore, the actual terms of support will vary by OEM; some OEMs might offer better provisions than others, and some points might be negotiable. Customers who buy their Datacenter system through a system integrator add a layer to the support process. You get the idea.

One thing I can say is that both Microsoft and its Datacenter OEMs are working hard to hold up their end of the deal. Together, they will have priority access to Datacenter experts and Microsoft Critical Problem Resolution and Quick Fix Engineering resources. They will use appropriate hardware to isolate and reproduce the problem and will provide speedy resolution. They will commit to being onsite (if necessary) within a specific, predefined time frame.

The Joint Support Queue's benefits are risk reduction for Datacenter sites, one support agreement and one support queue for hardware and software problems, Microsoft and OEMs leveraging their relationships with third-party vendors to resolve problems quickly, and problem-resolution facilities such as labs to reproduce bugs and fix them. If the vision even starts to become reality, Windows support will be much improved over what exists today, and Microsoft and OEMs will have set a precedent for better future support.

Hardware and Software Compatibility
Datacenter has two forms of officially sanctioned compatibility: the Datacenter Hardware Compatibility List (HCL) for hardware systems and devices, and the Certified for Microsoft Windows 2000 Datacenter Server logo for software applications. The former is a compatibility list—not a certification program—that Microsoft maintains. A hardware product must pass a rigorous Datacenter Hardware Compatibility Test (HCT) before Microsoft will add it to the Datacenter HCL. If a product passes the HCT and makes it onto the HCL, it also receives the Designed for Windows logo. You might hear someone refer to a piece of hardware as "certified" for Datacenter, but this really means that the hardware is on the Datacenter HCL.

The latter form of Datacenter compatibility is an official Microsoft-logo certification program, which the independent testing lab VeriTest executes and audits. Applications that meet Microsoft's specifications are certified and carry a Certified for Microsoft Windows 2000 Datacenter Server logo. For hardware, think HCL. For software, think Certified for Microsoft Windows 2000 logo.

To understand the entire Datacenter hardware compatibility and software certification process, you need to be aware of four main testing entities: the Windows Hardware Quality Labs (WHQL), the Datacenter HCT and its associated HCL, the cluster HCT and its associated HCL, and the Certified for Microsoft Windows 2000 Datacenter Server logo program.

Windows Hardware Quality Labs. WHQL is responsible for helping OEMs produce and test their hardware and software for best compatibility with Windows. WHQL (pronounced "wickel") maintains HCTs and HCLs for all Windows versions. The lab analyzes the results of HCTs and adds products that have passed to the appropriate HCLs. WHQL has been around since Windows 95, and managing the Datacenter tests is the latest addition to its responsibilities. For more information about WHQL, see http://www.microsoft.com/hwtest/default.asp and http://www.microsoft.com/windows2000/guide/datacenter/hcl/dchclprogram.asp.

Datacenter Hardware Compatibility Test. The HCT, a test suite designed to assess the stability of hardware running with Windows, is a self-test; OEMs run the HCT suite on their hardware in their labs. The HCT (now in version 9.x) is quite mature, and the Datacenter HCT requires that the OEM product run without failure for 14 days, exhibiting 100 percent availability during that time. (According to Microsoft, 100 percent availability during the test translates to 99.9 percent availability in the real world.) The goal is to verify that the hardware and any associated software are stable over time.

The HCT logs its test results in an encrypted file, which the OEM returns to WHQL after completing the test. WHQL then decrypts and interprets the results to determine whether the OEM's product has passed the test. In the event of a failure, WHQL works with the OEM to address the shortcoming. When a product passes, WHQL adds it to the HCL. The Datacenter HCL is an exclusive list—only a few systems have passed as of this writing. A product qualifying for the Datacenter HCL automatically qualifies for the standard Win2K Server HCL.

To view the Datacenter HCL, go to http://www.microsoft.com/hcl/default.asp, select System/Server DataCenter from the In the following types drop-down list, and click go. OEMs can download the HCT or order a CD-ROM at http://www.microsoft.com/hwtest/testkits.

Cluster Hardware Compatibility Test. The cluster HCT is specific for clustered solutions and is required if the hardware will be run in a clustered environment, which is the case for Datacenter hardware. WHQL adds to the cluster HCL those products that pass the cluster HCT. Microsoft clustering is supported only on HCL-qualified systems; that is, you will get support from Microsoft if the entire clustered system appears on the cluster HCL.

To view the cluster HCL, which isn't specific to Datacenter, go to http://www.microsoft.com/hcl/default.asp, select Cluster, and click go. For more information about the cluster HCT and HCL, see the Microsoft article "Microsoft Cluster Server Hardware Compatibility List and Testing" (http://support.microsoft.com/support/kb/articles/q224/9/71.asp).

Certified for Microsoft Windows 2000 Datacenter Server logo program. ISVs submit applications directly to VeriTest for testing. Datacenter applications bearing the Certified for Microsoft Windows 2000 Datacenter logo must meet a rigorous set of requirements, devised by Microsoft and outlined in the sidebar "Datacenter Application Specifications," page 56. You can download the specifications from http://msdn.microsoft.com/certification/download.asp. You can learn more about VeriTest's testing program at http://www.veritest.com/mslogos/ windows2000/win2k_datacenter.asp and view a list of certified applications at http://www.veritest.com/mslogos/windows2000/certification.

Typically, no HCT is required for ISV applications. However, applications that have kernel-mode device drivers, such as virus checkers and backup utilities, must undergo the Datacenter HCT before ISVs can submit them to VeriTest for certification. Additionally, VeriTest subjects applications to the Datacenter HCT again during the actual logo certification tests. Availability is paramount for Datacenter, and because poorly written kernel-mode device drivers can compromise system stability, drivers and their associated applications must demonstrate that they won't cause stability problems.

If you're an ISV, use the built-in Microsoft system services whenever you can. For example, if your application needs to monitor packets on the network, try to use the Windows Network Monitor APIs instead of creating your own device driver. The benefits are significant. You will likely develop your application faster because you won't have to write code that Microsoft has already written. Your application will represent a much lower risk to system stability. Finally, your application will proceed through the certification process faster, providing time-to-market advantages.

Taking on UNIX
Datacenter significantly raises the bar for Windows-based OSs running on Intel servers, and it places Win2K squarely in the competitive arena with powerful UNIX-based server solutions. Datacenter employs advanced and sophisticated technology to offer more scalability, higher availability, and better manageability than its Windows predecessors. For more information about Datacenter, see "Related Microsoft Web Sites," page 50.

In addition, Datacenter carries with it a unified support process, controlled hardware support, stringent software certification requirements, WHQL and OEM validation for kernel-mode drivers, and demonstrable stability. All these combine to provide the most formidable OS that Microsoft has ever developed.