Azure Cloud Storage Solution Supports Ultra-Demanding Workloads

Enterprise-class organizations have traditionally sought three things from the storage that services their most demanding workloads: low cost, high performance and high durability. Historically, however, it has been difficult, if not impossible, to find a solution that meets all three of these objectives. An on-premises storage solution that provides top-notch performance and extreme durability is not going to be cheap. Although an organization might be able to reduce its storage costs by moving to the cloud, it has been difficult to find a cloud storage solution that delivers both performance and durability.

Recently, however, Microsoft introduced a new class of Azure storage that might finally allow customers to move some of their most demanding workloads to the cloud. Azure Ultra Disk Storage, which was released to the public in August, is designed to provide extreme throughput and sub-millisecond latency.

Microsoft actually gave its customers a preview of Azure Ultra Disk Storage at the 2018 Ignite conference. During a demo, Microsoft’s Mark Russinovich showed that Azure could sustain a load of 160,000 IOPS with less than a millisecond of latency. Russinovich explained that Azure can achieve even greater levels of performance, and went on to demonstrate a “special server” with a “special disk” as a preview of what Microsoft has planned for the future. During that demo, Azure managed to sustain a load of over a quarter of a million IOPS with 1 millisecond latency. It is worth noting that both of these demos were based on the use of a single SSD.

The reason why Microsoft has been able to achieve these levels of performance is that it has completely rearchitected the way that its Azure-based Hyper-V servers communicate with back-end storage.

Unlike Azure Premium Storage, Azure Ultra Disk Storage does not utilize the Azure Blob storage cache. While the lack of caching might initially seem counterintuitive with regard to achieving extreme performance, bypassing Azure Blob Storage is one step in simplifying the path between Azure Hyper-V and the physical storage.

When Azure Premium Storage is used, Hyper-V leverages the Azure Blob Cache Driver. Additionally, storage traffic passes through a front-end server, followed by a series of partition servers and stream servers before ultimately reaching the physical storage device. In contrast, an Azure Hyper-V server that is configured to use Azure Ultra Disk storage communicates directly with a storage server by way of the Ultra Disk Virtual Disk client. The virtual disk client is fully aware of all of the storage mappings, which is why it is able to communicate directly with the storage server rather than depending on cache, load balancers or stream servers.

As previously noted, performance is only one of three factors that must be considered when architecting an enterprise-class storage solution for mission-critical workloads. The other two factors are durability and price.

It’s easy to assume that Microsoft would have sacrificed storage durability to achieve such extreme levels of performance in its Ultra Disk Storage solution. Somewhat surprisingly, however, Ultra Disk Storage incorporates the same Locally Redundant Storage technology that some of its other storage solutions use.

For those who might not be familiar with Microsoft’s flavor of Locally Redundant Storage--or LRS, as the company sometimes calls it--it is essentially a storage replication technology. Microsoft’s Locally Redundant Storage is designed to create three data copies within a single availability zone. Write operations are considered complete only when the data has been confirmed to have been replicated to the Locally Redundant Storage system. This added requirement makes the level of performance delivered by Ultra Disk Storage all the more impressive.

Of course, price is the other major consideration for any enterprise storage solution. Ultra Disk Storage is a premium solution that is priced at a higher rate than Microsoft’s other Azure storage offerings. It is worth noting, however, that Microsoft has designed Ultra Disk Storage to support dynamic (but not real time) performance tuning. This means that organizations can adjust both IOPS and throughput in response to changing workload requirements, possibly helping to drive down costs during periods of off-peak usage.

Click here for a list of the regions in which Azure Ultra Disk Storage is currently available. The solution is supported for use with DSv3 and ESv3 virtual machines.