How Netflix Retooled for the Cloud

Many companies with rapidly growing IT requirements are pondering their choices. Should they expand their data center space or port their operations to a cloud computing platform? The movie rental service Netflix is one of the companies that has chosen the cloud route. John Ciancutti, the vice president of Personalization Technology at Netflix, has written a blog post about the company's process of retooling its infrastructure to run on Amazon Web Services.

"One year ago, our cloud computing expertise was limited to research and some prior experience at other companies," Ciancutti writes. "Today Netflix is running one of the highest volume cloud computing deployments in the world."

Ciancutti's post reviews the reasons Netflix chose AWS. It wasn't a simple choice, as it involved "a Herculean effort" to re-write software and re-architect the company's service. A key factor was the difficulty of predicting the growth of the Netflix business and scale its infrastructure to keep pace. Ciancutti noted that  Netflix has revised its guidance for the number of year-end 2010 subscribers with three times over the course of the year.

"We are operating in a fast-changing and emerging market," he writes. "How many subscribers would you guess used our Wii application the week it launched? How many would you guess will use it next month? We have to ask ourselves these questions for each device we launch because our software systems need to scale to the size of the business, every time.

"Cloud environments are ideal for horizontally scaling architectures," he continued. "We don’t have to guess months ahead what our hardware, storage, and networking needs are going to be. We can programmatically access more of these resources from shared pools within AWS almost instantly."

Not every company has to deal with the rate of growth seen by Netflix. But the time and expense of provisioning new data center space has heightened the need for improved capacity planning. The emergence of the wholesale data center market has made it easier for companies to shorten the time frame in which they can add capacity. The emergence of modular data centers designs offers additional options to accelerate expansion on-site.

It's interesting to note that these changes have helped both enterprise users and cloud builders. But the planning challenges remain, and the Netflix experience offers valuable insight into how today's fast-growing companies are positioning their infrastructure to meet those challenges.