Storage Area Networks - 20 May 2002

Simplified storage management, narrow backup windows, and better application performance are some of the benefits

As e-business initiatives and growing demand for digital information force IT departments to expand storage resources while maintaining 24 x 7 access, managing storage assets efficiently becomes extremely important. Yet, managing storage that's directly attached to servers becomes increasingly difficult, costly, and time-consuming as the number of servers grows. Thus, IT departments are looking to Storage Area Networks (SANs) and Network Attached Storage (NAS). Both SANs and NAS simplify the storage-management task by consolidating server storage into central storage arrays that you can manage from one administrative console.

In contrast to NAS systems, which connect to your LAN and provide file-level access for client systems, SANs are dedicated 1Gbps or 2Gbps storage networks that connect disk arrays, servers, tape libraries, and other peripherals to a switching fabric or hub. SANs provide block-level access to data residing on shared storage arrays. Moving storage traffic onto a dedicated high-speed network can narrow backup windows and improve application performance by reducing contention for LAN bandwidth. And SAN management software can easily perform tasks such as reallocating storage space among servers without interrupting access to data. In addition, many NAS vendors offer products that can share SAN-based storage (for more information about such NAS products, see "Storage Networking," February 2002, InstantDoc ID 23563).

The Proven Standard
SANs are built around Fibre Channel technology because of its high speed (as fast as 2Gbps) and reliability compared with Ethernet networks. Companies such as Compaq, EMC, Hewlett-Packard (HP), IBM, and Nishan Systems provide full SAN solutions, frequently including networking components from third-party vendors such as Brocade Communications Systems, McDATA, Gadzoox Networks, QLogic, Adaptec, and Emulex. When you buy a SAN from a major solution provider, you can be confident that software and hardware will work as expected. However, assembling a SAN from various suppliers' components will require more effort because interoperability of third-party Fibre Channel SAN components is a work in progress.

Fibre Channel SANs are expensive and complex to implement and manage, so they've been implemented primarily by large enterprises that have inhouse technical expertise and challenging storage-management environments. A relatively large SAN that supports hundreds of users and has several Fibre Channel directors (switches with many ports and redundant backplanes) and storage arrays can cost $1 million or more. Even a small SAN that has an arbitrated loop hub and one storage array and supports no more than 20 servers can cost a few hundred thousand dollars.

Interoperability has also been a problem. Until recently, assembling a multivendor Fibre Channel SAN was extremely difficult because vendors have interpreted Fibre Channel standards differently and have used proprietary features to their competitive advantage. For example, you couldn't assume that different vendors' Fibre Channel switches would interoperate or that a particular vendor's SAN management software would work with another vendor's switches, host bus adapters (HBAs), or storage arrays. But as I explain later, that situation is beginning to change.

An IP-Based Approach
Concerns about interoperability, the cost of Fibre Channel SAN implementations, and the need for highly trained IT staffs and specialized Fibre Channel network-management tools have spurred interest in the potential of IP-based storage networks, particularly as the price of Gigabit Ethernet switches has dropped. As a result, storage vendors are lining up to support the pending complementary protocol standards from the Internet Engineering Task Force (IETF): Internet SCSI (iSCSI), Fibre Channel over IP (FCIP), and Internet Fibre Channel Protocol (iFCP).

The iSCSI protocol lets IP networks carry SCSI commands and storage data, making IP-based SANs feasible. FCIP lets IP networks carry Fibre Channel control codes and data, so you can use IP links to connect geographically separate Fibre Channel SANs, as Figure 1 shows. The iFCP protocol lets Fibre Channel SAN components use lightweight gateways to connect to IP networks.

Depending on the level of performance you need, your budget, and how you use your existing network, you can add a storage array to an existing Gigabit Ethernet backbone (after installing iSCSI HBAs in each server) to implement an iSCSI SAN, or you can create a totally separate back-end network, as most enterprises do in their Fibre Channel installations. Companies that have a Fibre Channel SAN can also implement an iSCSI SAN and link the two SANs with a storage router that supports both protocols, as Figure 2 shows.

The three new protocols should be ratified later this year, and IBM, Nishan Systems, Cisco Systems, and others have already introduced storage products based on them. Whether first-generation iSCSI products will interoperate smoothly with one another remains to be seen, but vendors insist that the maturity of TCP/IP technology and the testing and refinement work being conducted by the Storage Networking Industry Association (SNIA) IP Storage Forum's member companies will ensure a high level of interoperability.

The Solution Is in the Cards
IP SAN technology is an attractive alternative to Fibre Channel. However, IP technology presents TCP/IP and iSCSI protocol overhead problems that vendors still need to overcome. Server CPUs currently handle the overhead on Gigabit Ethernet LAN segments, but a SAN's high-volume, block-data traffic increases this processing overhead and can substantially slow server performance.

Several vendors have introduced iSCSI HBAs with TCP/IP off-load engines to address this problem. Some products, such as Adaptec's ASA-7211 iSCSI Adapter, off-load all processing overhead; others, such as the Alacritech 1000x1 Single-Port Server and Storage Accelerator, partially off-load processing and relegate control and error handling to the server's CPU. These design choices stem from vendors' differing opinions about what percentage of dropped packets—and the corresponding CPU overhead that retransmitting them causes—will be incurred at a SAN's high traffic volume. Which approach is better remains to be seen. Pricing for the Adaptec card wasn't available at press time, but the Alacritech product lists for $999—about the same price as a Fibre Channel HBA. Most such cards also combine Gigabit Ethernet NIC functionality. Depending on whether a vendor's lineage traces to storage or networking, these cards are called HBAs or NICs.

Bringing Fibre Channel to the Mainstream
Although small and some midsized organizations will likely purchase IP-based SANs, some Fibre Channel SAN vendors are delivering new products that make Fibre Channel solutions more attractive to these higher-volume markets. For example, Compaq's recently introduced Modular SAN Array 1000 (MSA1000) storage array is relatively inexpensive and easier to get up and running for companies that use Compaq's ProLiant servers than most other Fibre Channel storage arrays are. Because this 2Gbps Fibre Channel array can use SCSI disk drives from ProLiant servers, you might not need to purchase much additional storage. And you can move disks from ProLiant servers to the MSA1000 storage array with data intact, letting you reconfigure the storage array without doing a backup and restore. The MSA1000, which has a starting price of $9995 without disks, has an internal capacity of 1TB; you can use external storage cabinets to expand the capacity to 3TB. This storage array is also available with an internal 6-port 2Gbps Fibre Channel switch for an extra $5295.

QLogic is also attempting to provide a simplified, low-cost Fibre Channel SAN solution for small to midsized businesses with its $9999 SAN Connectivity Kit 1000, which bundles compatible components and software. The kit contains an 8-port 1Gbps Fibre Channel switch, four 1Gbps fiber optic HBAs, fiber optic cabling, four 1Gbps Gigabit Interface Converters (GBICs) to convert electrical signals into optical ones, switch and HBA management software, and SAN management software. The kit supports a variety of storage arrays, including models from EMC, Hitachi, and Compaq, and QLogic offers upgrades of individual components. Although you need to provide the storage array, which is likely to be the most expensive component, the kit lowers the cost and reduces the complexity of implementing a Fibre Channel SAN.

Working Together
Although Fibre Channel interoperability continues to be a concern, vendors are removing compatibility roadblocks, and those efforts are beginning to pay off. Over the past year, most Fibre Channel—switch vendors have released products that support the ANSI's Fibre Channel Switch Fabric 2 (FC-SW-2) standard. This standard's common interoperability mode seems to have solved switch interoperability problems. Similarly, vendor compliance with SNIA's Fibre Channel API standard has eased HBA interoperability difficulties.

Many vendors are certifying their SAN management applications with third-party hardware, and most large vendors will test their products with specific third-party products upon customer request. But SAN management applications often provide only a basic level of interoperability; if you want to use all the software's functions, you typically need to buy your hardware from the same vendor.

EMC and Compaq are an exception to that rule. These companies cross-license their SAN APIs to provide better interoperability between their products. As a result, EMC's ESN Manager software (starting at $18,000) supports storage array zoning on Compaq's StorageWorks arrays, in addition to supporting discovery and reporting features on HP's XP512, IBM's Enterprise Storage Server (ESS) Shark, Hitachi's Freedom Storage Lightning 9900 Series, and Sun Microsystems' StorEdge 9900 storage arrays. At press time, the support for the Compaq arrays didn't include the device- masking capability that EMC provides for its products. EMC says it's in discussions with other vendors to establish API-sharing agreements but wouldn't disclose which vendors.

SAN vendors and SNIA have also been working together to assemble, test, and support multivendor SAN solutions under the auspices of SNIA's Supported Solutions Forum (SSF). Formed in June 2001, SSF now has 32 member companies. These companies create and test configurations, then register them with SNIA. After confirming that a configuration meets SNIA criteria, SSF posts it on the SNIA Web site (http://www.snia.org). A posted configuration must include products from at least four vendors; many configurations provide multiple product choices within a particular category (e.g., a configuration might specify two manufacturers' storage arrays within the same data zone). All vendors involved in the configuration must agree to support the entire configuration. Thus, you have one point of contact even if your configuration has problems that might involve more than one vendor. If you're attempting to build a multivendor SAN from scratch or are adding third-party products to an existing SAN, you should consider the SSF's posted configurations.

The goal of having one management application that supports many vendors' SAN hardware will move one step closer to reality when hardware vendors begin including support for version 2.7 of the Distributed Management Task Force's (DMTF'S) Common Information Model (CIM) in their products. Switches and storage arrays that support CIM 2.7 should begin appearing by the end of 2002 and will communicate their configuration and capabilities to the management software in a commonly understood format. CIM 2.7 will also define a standard way of managing hardware products. After vendors implement the standard, most management software will support new hardware products out of the box. The CIM 2.7 standard is gaining broad support from hardware and software vendors because it will eliminate much custom coding needed to support new hardware. However, it might take 2 years or more for all vendors to add this support to their products.

At the Crossroads?
Small businesses that have neither deep pockets nor IT staffs with Fibre Channel expertise will probably be the early adopters of IP storage technology. Large firms typically don't adopt new technologies quickly, and those that have already implemented Fibre Channel—based SANs probably will expand them as needed because Fibre Channel SANs provide outstanding performance and reliability.

However, not all enterprises need 2Gbps performance, and if the first generation of IP storage products proves to be interoperable and reliable, midsized businesses will face a tough decision between Fibre Channel and iSCSI. Although Fibre Channel is a proven technology, iSCSI will cost less to implement even if you create a totally separate storage network. What's more, IP networks offer high quality, security, and management capabilities that aren't yet available with Fibre Channel.

^^^^ ^^^