WANdisco Service Enables Petabyte-Scale Data Migration to Azure

WANdisco and Microsoft have joined together to create a native Azure service that allows organizations to move very large amounts of data to the cloud without interrupting the course of business.

WANdisco LiveData Platform for Azure relies on WANdisco's Distributed Coordination Engine (DConE), which during data migration to Azure enables active-active replication with no single points of failure or scalability bottlenecks, according to Daud Khan, WANdisco's vice president of corporate development. With this engine, the servers that make up a distributed system can be on the same local area network or connected over a wide area network.

In addition, DConE does not alter the behavior of the underlying application and maintains the illusion of a single server system performing at LAN speed in a distributed environment where the servers are actually thousands of miles apart, Khan explained. This maximizes productivity and reduces costs by enabling consistent real-time data access and collaboration across a globally distributed organization.

The result is a system that allows for petabyte-scale cloud migration without interrupting business, something that has been virtually impossible to accomplish, said Merv Adrian, research vice president for data and analytics at Gartner.

"Because the volume is so large for most organizations, they have had two choices: either make a completely separate version and not spin it up until you're finished, or deal with the fact that during the migration, you won't be able to work on it," he said. "So for many organizations, the requirement to create another copy before you even move it and then have to move it was complex, challenging, disruptive, risky and expensive."

The system's synchronization capability means that a change made at the source end while the transfer is live will be captured and applied at the target end, allowing business to proceed as usual on both sides.

To simplify the process of data migration to Azure Data Lake Storage (ADLS), WANdisco and Microsoft are providing two tools: LiveData Migrator for Azure and LiveData Plane for Azure. These services automate active-active data plane consistency at scale between an on-premises big data Hadoop environment and Azure Data Lake Storage Gen2, according to WANdisco.

LiveData Migrator for Azure simplifies data migration to Azure Data Lake Storage by using a single scan of the data and ensuring any changes in the source destination is reflected in ADLS as the migration proceeds.

With LiveData Plane for Azure, organizations can create a two-way consistency between source and destination, creating a true hybrid cloud, where applications in the cloud can read and write data and those writes will be reflected on-premises and vice versa. "This removes the need for a large cutover between environments and removes data risk during a migration," Khan said.

While the tools themselves are significant, Adrian said he believes the bigger deal is the nature of the relationship between WANdisco and Microsoft.

"The fact that they have created an embedded, native Azure service is a big deal for organizations concerned about auditing, would like the billing to be seamless and want to integrate the whole thing into their existing security infrastructure," he said. "It sort of takes the 'D' out of 'DIY.'"