VMware Cloud CTO: Launching VMware on AWS Has Been a Heavy Lift

Launching a VMware cloud in Amazon Web Services data centers has been a very different kind of project from what AWS and VMware engineering teams are used to.

Giving customers the ability to spin up VMware virtual machines on Amazon servers, a VMware-operated service launched last August – almost a year after the companies announced their partnership – took a lot of learning and deep architectural changes on both sides of the deal, Kit Colbert, a VMware VP and CTO of the company’s Cloud Platform business unit, said in an interview with Data Center Knowledge.

“This is not just something we threw together and put out there,” he said. “This required actually really deep modifications on both sides.”

The partnership between the world’s largest public cloud provider and the leading data center software company has a lot of potential. Like others in their field, both are courting business with large enterprise users, many of whom are transitioning to a modernized, cloud-first type of infrastructure.

VMware is deeply entrenched in enterprise data centers around the world. Its server virtualization software is ubiquitous, and its cloud infrastructure software suite is popular among enterprises building and operating private clouds.

Many companies want to leverage the scale and capabilities of Amazon’s public cloud infrastructure while keeping some of their computing capacity in-house, and the promise of seamless integration between the on-premises VMware environments they know and love with AWS is a powerful one. The two vendors are essentially promising a hybrid-cloud easy button.

But integration between the two on the backend has been a heavy lift, which means the full impact of the partnership is going to take some time to unfold. Their architectures couldn’t be more different from each other, so both companies have had to tweak their technology and create new operational processes.

“We’re learning how to operate a cloud essentially, and so there’s a lot of work we need to do there,” Colbert said about VMware, which has operated a cloud business before but sold it last year to the French cloud provider OVH.

Invasive Surgery

To launch VMware on Amazon’s cloud infrastructure AWS engineers had to change “how they actually architect their data center,” Colbert said. “Because they’ve never actually had anyone else’s software run on their bare metal. It’s always been on top of EC2.” (EC2, or Elastic Compute Cloud, is Amazon’s popular cloud VM service.)

Traditionally, Amazon’s infrastructure has been virtualized using the open source Xen hypervisor, but the company has been transitioning to a new custom distribution of KVM (also an open source hypervisor) since last year.

Running a VMware cloud means using VMware’s ESXi hypervisor of course. And everything in the AWS architecture, including network, security, physical-server provisioning, and interaction with EBS (Amazon’s Elastic Block Storage), has had to be modified, often at the hardware level, Colbert said.

Just from the security perspective, “you now have a host with someone else’s software on it. That’s actually a huge potential security issue, if they didn’t lock that down right,” he said. “Obviously they trust us, but at the same time, they shouldn’t trust us too much, right? So, you’ve got to have the right sorts of guards in place.”

To adapt its physical-server provisioning processes, AWS engineers “had to create a new API that we call when we want a new bare-metal server,” Colbert said. Amazon has a complicated capacity planning model, which maximizes profitability by enabling its engineers to turn new servers on and put them to work as soon as they’re racked, he explained. “They’re constantly racking servers for EC2; they’re very good at this.”

Groundwork Laid for Bare-Metal Cloud Service

It’s going through this integration process that allowed AWS to introduce its bare-metal cloud service. “They actually took what we worked on together, and they released that as a stand-alone service,” Colbert said.

The service went into general availability this May. It enables customers to provision dedicated physical servers onto their networks through the same EC2 interface they use to spin up cloud VMs.

The technology underneath may not be exactly the same as the technology used for VMware on AWS, but “conceptually, it’s identical,” Colbert said.

Essentially, integration with VMware allowed AWS engineers to work through a lot of the big challenges involved in building a bare-metal cloud service, so actually launching one wasn’t a big leap at that point.

Integration Work Far from Over

VMware has partnerships with other cloud providers, but none at the same depth as AWS. Even if VMware is interested in striking similar partnerships with the likes of Microsoft Azure, Google Cloud Platform, or IBM Cloud, it will not happen soon, if only for the amount of complexity involved in integrating another company’s architecture with a hyperscale cloud infrastructure designed from the ground up to support something entirely different.

Customers have been asking for similar integrations with Azure and others, but Colbert’s team still has a long way to go before that can enter the realm of the possible. “The job there is not done; we have a lot of stuff to do to really complete the vision,” he said. The goal now is for the service to reach global coverage (it’s currently available in Northern Virginia, Oregon, Frankfurt, and London) and integrate deeply with advanced AWS services.

“We really don’t want to dilute our focus too much by doing too much too quickly.”