Virtual Interface Architecture - 27 Nov 2000

Recognizing a need for efficient, high-speed, low-latency, server-to-server communications, Compaq, Intel, and Microsoft developed the Virtual Interface (VI) Architecture specification and published version 1.0 in 1997. The vendors intended the specification to improve the performance of distributed applications, especially for database I/O and clustered-system communications.

Implementations of VI Architecture that work through Windows NT's Winsock protocol stack have existed for more than a year. But Microsoft's first true native support for VI Architecture appears in Windows 2000 Datacenter Server (in the form of Winsock Direct) and in SQL Server 2000 Enterprise Edition's native support for two implementations of VI Architecture—Giganet's cLAN and Compaq's ServerNet II NICs—running under any version of Win2K Server. Native VI Architecture support means that SQL Server network I/O bypasses the standard Winsock protocol stack, which improves overall system performance. SQL Server 2000's support for VI Architecture meets the need for efficient application-server to database-server communications and the need for links among database servers in federated database implementations.

Both the cLAN and ServerNet II implementations of VI Architecture support 1.25-gigabit full-duplex speeds across a switched network architecture. CPU-efficient, low-latency data transfer is the key benefit of using VI Architecture for clusters and tiered database applications. The improvements result from a network adapter and driver architecture that eliminates much of the kernel-mode processing associated with the Ethernet architecture.

The software foundation for VI Architecture support is VI's Kernel Agent, a kernel-mode driver that creates VIs and manages OS resources such as memory on behalf of the VI. VI's User Agent, operating in user mode, is the interface that application programs use to communicate with the Kernel Agent and the VI. When an application opens a connection to a VI-compliant network adapter, the Kernel Agent creates a VI, which is a logical network adapter dedicated to that application. The Kernel Agent allocates nonpageable, reusable memory buffers to each VI. These fixed memory buffers eliminate the delays and kernel-mode CPU activity that memory allocation and address translation cause during I/O activity. The buffers are also a basic resource for VI Architecture's support of Remote Direct Memory Access (RDMA) data transfer. With an RDMA Read or Write, one computer can initiate data transfer directly from or to the memory of another computer, eliminating CPU involvement on the receiving computer during the I/O operation. VI Architecture also supports traditional send-and-receive data transfer; each vendor's implementation of VI Architecture uses a predetermined algorithm to determine whether send-and-receive or RDMA data transfer is more efficient for a particular request. The algorithm weighs the overhead associated with copying data between send-and-receive buffers against the overhead of allocating RDMA buffers. RDMA wins for large message sizes (e.g., 64K).

The VI Architecture specification supports three reliability levels for data transfer. At the most reliable level, Reliable Reception, the adapter considers the write operation complete when the destination interface acknowledges that the buffer has received the data. At the Reliable Delivery level, the adapter considers the write operation complete when data leaves the local system. At the Unreliable Delivery level, the adapter signals the application that the I/O is complete when it starts the I/O operation, with little monitoring of the result. In general terms, Unreliable Delivery compares to the reliability that the IP network stack supports and Reliable Reception compares to the reliability of the TCP protocol stack.

Vendors can choose not to implement some of the optional capabilities that the VI Architecture 1.0 specification defines. For example, RDMA Read support is optional, as are the two most reliable levels of data transfer: Reliable Delivery and Reliable Reception. cLAN implements the equivalent of Reliable Reception in hardware and generates the equivalent of an RDMA Read by causing the other end of the connection to initiate an RDMA Write. ServerNet II implements all three levels of data transfer as well as RDMA Read and Write.

The cLAN and ServerNet II support in SQL Server 2000 includes no code to support the TCP/IP or Winsock protocols. SQL Server communicates directly with the User Agent, which during I/O activity communicates directly with the VI created for the connection. Datacenter's Winsock Direct, which is designed to support current and future network architectures that implement data-transfer reliability in hardware, provides an additional driver layer, called the Winsock Provider, between the application and the VI interface. This layer lets existing applications written to the Winsock or TCP/IP interface take advantage of VI's improved performance. The cost of this additional layer compared to "raw VI," according to Microsoft, is about a 5 percent loss in throughput and 5 percent increase in CPU utilization—a small price to pay for the near 100 percent level of application compatibility that Microsoft claims the Winsock Provider gives you.

VI Architecture improves application performance in a couple of ways. First, the reduced instruction-path length—the number of machine-level commands that you need to execute—translates directly into faster I/O completion (lower latency). Second, using fewer CPU cycles for network I/O leaves more CPU cycles free for use by other applications on the server.

For more information about this specification, see the VI Architecture Web site (http://www.viarch.org), which Compaq, Intel, and Microsoft established. You can find information about cLAN and other VI-related links at http://www.giganet.com. For information about Compaq ServerNet II, go to http://www.servernet.com.

You'll want to track a couple of other high-speed data-transfer initiatives. Network Appliance is developing the Direct Access File System (DAFS) protocol, which will leverage VI Architecture and future RDMA networking technologies to provide standards-based file-sharing in Windows and UNIX environments. And the InfiniBand Trade Association—which Compaq, Dell, Hewlett-Packard (HP), IBM, Intel, Microsoft, and Sun Microsystems formed—is developing the specification for a general-purpose, switched-fabric I/O architecture to support both local PCI-like cards and remote fibre-channel-like storage networks. VI is a step along the path to InfiniBand. But whereas InfiniBand isn't expected to be commercially available until late in 2001, VI Architecture is here today.