Apple Discusses Going Cloud Native and the Growing Pains

In her keynote at the recent virtual KubeCon + CloudNativeCon North America hosted by The Linux Foundation's Cloud Native Computing Foundation, Alena Prokharchyk, software engineer with Apple, discussed her company’s adoption of a cloud-native ecosystem for application development and deployment -- and lessons learned along the way. She said the move was made to support the tech giant’s scale and use cases more efficiently with a focus on applications for modern, cloud environments.

Apple migrated from Apache Mesos to Kubernetes, Prokharchyk said, drawn to the latter platform’s focus on security, privacy, multitenancy, and scale. She said Apple used Mesos for more than five years with an internally built container orchestrator called Jarvis. “It had a good reputation as a scheduling platform, and it scaled well,” Prokharchyk said. Yet that container orchestrator became a bottleneck for responsibility delegation, she said, which led Apple to explore other options.

In addition to using Mesos, she said Apple used other solutions for its compute management. The migration from Mesos presented Apple with an opportunity to consolidate those solutions into a unified platform, Prokharchyk said. “We found Kubernetes to be the obvious winner as an orchestrator.”

The generic, pluggable nature of Kubernetes made it the right fit for Apple’s teams, she said. “The decisions on what provider to choose for core components like storage, runtime, and network were no longer life or death decisions,” Prokharchyk said. Unlike working with Mesos, she said plug-in choices in Kubernetes could be reevaluated without refactoring the entire system.

Another reason why Apple developers embraced the migration, she said, was the ability to extend Kubernetes APIs and core functionality while leveraging custom controllers. The shared input from other Kubernetes users was also helpful for sorting out the migration. “The Kubernetes community was a huge asset to the platform,” Prokharchyk said. “Its transparency and power . . . provided a great level of comfort to developers.”

Though the prospects for Kubernetes were appealing, some training and education was necessary for its adoption, she said. At the time Apple undertook the migration, some of its teams were already using Kubernetes clusters while some personnel had never worked with Kubernetes before, Prokharchyk said. “When a change of this magnitude happens, it is important to be honest about the tradeoffs that need to be made for a successful migration,” she said. “It is a known fact that the Kubernetes learning curve is steep.”

In addition to upskilling personnel, old platform features had to be reevaluated to see if their designs aligned with cloud-native best practices, Prokharchyk said. Those old platform features also needed to be assessed to see if they were still relevant to the new platform. “We had to accept the fact that there would be a need for change to the current processes and people would have to adapt,” she said.

Apple provided training resources and other support to get developers up to speed for the migration, she said, in order to make the adoption of the new platform more successful. “Our target goal is to make the majority of Apple workloads run on Kubernetes,” Prokharchyk said.

The company also sought to maintain its end-user focus while building the platform, she said, which required understanding their needs while to planning the new hardware and software infrastructure. “Our responsibility as platform developers is to provide a scalable orchestration layer with secure resource isolation and reliable scheduling,” she said.

A technology shift of this magnitude also means organizational changes, Prokharchyk said. “You need to change everything and everyone, from managers in finance to the dev team,” she said. “It is a long road with a lot of growing pains.” For example, Apple’s security team grew to take on additional responsibilities, Prokharchyk said, which included the complex task of ensuring security in a multi-tenant cluster.

Through its journey to a cloud-native platform for development, she said Apple learned to embrace the Kubernetes community’s best practices and shared learnings. This helped make the migration work at scale despite the disruption it meant for developers, Prokharchyk said. “Migration is not something everyone wants to do,” she said. “Those who invest in modernizing their application platform can benefit in the future by adopting more cloud-native technologies on top of the foundation.”