Microsoft’s Storage Spaces Direct: How It Differs from Other HCIs

In the last blog we focused on HCI products that had disaggregated architectures, meaning they have dedicated storage and compute nodes instead of combining both resources in every node. Another product that belongs on that list is Microsoft’s Storage Spaces Direct (S2D). Originally released with Windows Server 2016 Enterprise Edition and removed from a release in October 2017, S2D is part of Windows Server 2019 Enterprise Edition. S2D runs in the Windows kernel, handling the storage virtualization tasks in the operating system.

Microsoft doesn’t sell this as an HCI appliance directly, instead partnering with server vendors to offer HCI solutions under the Windows Server Software Defined (WSSD) program. In this blog we’ll refer to Storage Spaces Direct as the HCI solution, but technically each vendor in the WSSD program names its own products. For example, Dell’s is called Storage Spaces Direct Ready Nodes, HPE calls its HPE WSSD, and Fujitsu’s solution is called PrimeFlex for Storage Spaces Direct.

While most companies prefer to buy HCI products as complete solutions, with S2D users can “roll their own” HCI as well. Since Hyper-V is included, S2D can turn a group of servers with the appropriate resource configurations running Windows Server into an HCI cluster.

S2D can operate in either a “converged" mode, where storage and compute functions are on separate nodes, or in “hyperconverged" mode, where both functions run on the same node. While the vast majority of deployments are in hyperconverged mode, S2D can actually be used to create a scale-out storage system sharing storage pools on the cluster with external servers.  

HCIs incorporate software-defined storage as a virtualization layer, allowing them to create pools of storage, either block- or file-based, that are shared with the applications running on each node. The virtualization layer enables the cluster to be easily expanded by adding more nodes and can build in different kinds of resiliency, most often creating multiple copies of data blocks and distributing them to different nodes. 

Most HCIs run their software-defined storage software as a VM on each node, which requires a hypervisor on each node as well. Disaggregated architectures do it a little differently. They run a storage virtualization layer in the operating system, not as a VM. With Storage Spaces Direct, it is embedded in the Windows Server operating system, meaning storage processing doesn’t have to go through the hypervisor stack, which can improve performance.  

Other disaggregated HCIs can run a hypervisor on their compute nodes to support multiple workloads, which requires the appropriate hypervisor licenses. But HCIs built with Windows Server 2019 come with a Hyper-V license, which is already part of the OS. This can provide some cost savings depending on the licensing arrangement. 

Another difference in S2D is internode connectivity. Most HCIs use iSCSI between nodes, but S2D is based on the SMB direct feature of SMB 3.0, which uses RDMA over iWARP or RoCE with 10/40GbE. Together with the embedded architecture, S2D promises some performance improvements over other HCIs.

Like many other HCI products, companies can set up their own HCI clusters by following the hardware compatibility specs, but most are more interested in buying complete solutions. In 2017, Microsoft launched the WSSD program, which certifies hardware configurations to create an HCI. The WSSD stack includes Hyper-V hypervisor, Storage Spaces Direct software-defined storage, software-defined networking (SDN) and Windows Admin Center for VM and infrastructure management.

Currently, WSSD includes all major hardware vendors except Cisco. In this way WSSD is somewhat like the vSAN ReadyNodes program.

At Evaluator Group, we first got a glimpse of Storage Spaces Direct in early 2016 (it was released with Windows Server 2016) and were impressed with the architecture of this software-defined storage product. It had disaggregated storage and compute (one of the first), it was embedded in the server OS and included advanced RDMA connectivity as well as many of the storage services that IT has become accustomed to.

In 2016 the HCI market was maturing, but we didn’t see the disruption we expected from this product. Part of that was probably due to the strength of VMware as the hypervisor standard, but also the lack of a strong support program for getting hardware vendors on board with Windows Server 2016 and S2D as an HCI solution. Microsoft also apparently had some technical issues with S2D and removed it from a Windows Server 2016 release in October 2017.

The latest version of S2D in Windows 2019 (which adds better performance, improved dedupe, at-rest encryption and larger maximum capacity) and the WSSD program may address these issues, as now more than a dozen vendors are selling HCI solutions using S2D. In terms of cost, Microsoft has an advantage with Hyper-V being included in Windows Server OS. Nutanix reports increasing success converting users to its AHV hypervisor, to save on licensing costs. Perhaps Microsoft can tap into that same demand.

The performance that S2D brings may open this product to some more extreme use cases, but the real opportunity for Microsoft is to show that performance can drive up VM density (the number of VMs that can be supported on each node) for a given workload mix, which will drive down cost proportionally. For users this means fewer nodes to buy to support a given set of workloads and a smaller cluster to manage.