5 Reasons Not to Use Kubernetes Distributions

There are two main ways to go about deploying Kubernetes. One is to adopt one of the dozen-or-so Kubernetes distributions that software companies have built. The other is to install “vanilla” Kubernetes yourself using the official Kubernetes source code.

Conventional wisdom is that the latter approach is rarely a good idea. Most people would tell you that, unless you are a Kubernetes developer and want to play around with the core code, you’re better off using a Kubernetes distribution that offers a turnkey installation and deployment experience.

That may have been sound advice in the past (and, admittedly, it’s advice I’ve given to people myself). But I increasingly think that there are good reasons to stick with upstream, vanilla Kubernetes, rather than using prebuilt Kubernetes distributions. Here’s why.

What Is Vanilla Kubernetes?

Vanilla Kubernetes refers to the plain, unmodified version of Kubernetes that you would get if you were to download and install the source code.

It’s called vanilla Kubernetes because there is a decades-old tradition in the software world of applying the label “vanilla” to any application or platform that is deployed using its official version, without any modifications. Thus, you might also hear folks talk about “vanilla Linux,” for example, when referring to a Linux kernel that is built purely from the official Linux kernel source code, without the modifications that are often made to the kernel when it is incorporated into a specific Linux distribution.

The opposite of vanilla Kubernetes are Kubernetes distributions, such as Rancher, Red Hat OpenShift or a cloud-based Kubernetes service like Amazon EKS. These distributions take the core open source Kubernetes code and integrate it into broader platforms, which usually include management, monitoring and security tools that are not part of Kubernetes itself. Many of them also offer installers to make the Kubernetes installation process very easy.

When Kubernetes Distributions Don’t Make Sense

Kubernetes distributions certainly have their benefits. I’m not here to tell you that no one should ever use them. But I’d like to push back against conventional wisdom that suggests that 95% of Kubernetes use cases are best served by a distribution, as opposed to vanilla Kubernetes.

Here’s a look at reasons why vanilla Kubernetes may work better for you than a distribution.

1. Kubernetes is not Linux (it’s much less complex).

For starters, many arguments about why you should avoid vanilla Kubernetes hinge on the idea that Kubernetes is just like Linux, in that it’s a complex platform that would be way too hard for most people to build and install from scratch.

The reality is that the similarities between Linux and Kubernetes are limited. Yes, both are complex open source codebases. And installing both from scratch does require some time and effort.

Compared to Kubernetes, however, Linux is far more complex. To set up a usable Linux-based operating system from scratch, you’d have to build and install dozens, if not hundreds, of individual components.

In contrast, Kubernetes consists of about a dozen components, some of which are optional. You could realistically sit down and install each one in a day. The same can’t be said of all of the utilities and applications that go into a Linux distribution.

It’s worth noting, too, that part of the reason why you’d opt to use Linux distributions instead of installing Linux from scratch is that distributions make it easier to keep your software up-to-date via package managers. Easier updates may be a benefit of Kubernetes distributions, as well, but because there are many fewer components to worry about updating in Kubernetes, the value of this functionality is much less important.

2. Kubernetes distributions create ecosystem lock-in.

If you use a Kubernetes distribution, you get locked into the ecosystem surrounding that distribution.

This does not necessarily mean you are tied to a specific company’s software; in many cases, you can easily adopt third-party, open source add-ons for your Kubernetes deployment even if you use Kubernetes distributions. And many of the required components within most Kubernetes distributions are themselves open source.

But, in most cases, installing Kubernetes with a distribution instead of using vanilla Kubernetes will lead to dependency on certain core tools or dependencies. If you use OpenShift, for instance, you’ll be tied to its various components. If you run a cloud-based Kubernetes distribution like AWS EKS or Azure AKS, you’re stuck within that cloud provider’s ecosystem.

If you install vanilla Kubernetes, you’ll have more freedom to pick and choose the components you want to use to build your deployment. That’s one of the benefits you get in return for the extra effort required to install vanilla Kubernetes.

3. Kubernetes distributions come at a cost.

There is a cost factor worth considering, too. Most Kubernetes distributions are developed by commercial companies and cost money to run at scale. (Some do offer free tiers for small-scale deployments.)

There is, of course, a monetary cost associated with vanilla Kubernetes, too. The source code may be free, but the time that your engineers spend setting up and managing it is not. This argument has long been used to make the case for choosing paid versions of open source software instead of taking a DIY approach.

When it comes to Kubernetes, though, I’m not sure that the staffing costs associated with vanilla Kubernetes are necessarily higher than those of a Kubernetes distribution. Managing Kubernetes is pretty complex no matter how you deploy it, and you’ll end up having to pay engineers a lot of money to monitor and maintain your deployment even if you use a distribution.

Plus, given that (as noted above) Kubernetes has only about a dozen components, installing and setting it up will probably not take most teams an inordinate amount of time. In this sense, too, there is less cost value in choosing a distribution that streamlines the installation process.

4. Kubernetes distributions may die.

Kubernetes remains a very young platform. Most of the mainstream Kubernetes distributions available today have existed for only a few years. (Or, at least, they have only been Kubernetes-based for that time; some distributions, like OpenShift, originated as platforms that were not initially built around Kubernetes.)

Some Kubernetes distributions, like Kontena Pharos, have already failed. Several others, like Kublr and Rancher, are maintained by startups that may or may not be around five or 10 years from now. And even the distributions from major enterprises, like IBM (which is now the owner of Red Hat OpenShift) and AWS, could be killed off with little notice.

It’s a safer bet that the open source Kubernetes project will endure for a long time. If you build your Kubernetes strategy around a distribution, you’re at comparatively greater risk of your distribution disappearing and being forced to migrate to a new distribution.

5. Vanilla Kubernetes is getting easier to install and maintain.

Finally, Kubernetes itself is becoming more user-friendly with each release.

Like most open source projects, Kubernetes was pretty rough around the edges early on. It suffered from a litany of bugs, features that were not fully documented and integrations that often only half-worked. Faced with these issues, it was attractive for many teams to choose a Kubernetes distribution from a vendor who resolved many of these problems.

Today, however, Kubernetes has come a long way. It certainly has its bugs, but they are generally less critical than they were in the past. Configurations are easier to maintain. The Kubernetes documentation is exemplary for its organization, navigation and clarity.

Conclusion

To be sure, there are lots of good reasons for using a Kubernetes distribution instead of vanilla Kubernetes. Kubernetes distributions require less effort and expertise overall. But they also come with higher costs, less flexibility and less assurance of long-term availability.

Before you go and install a Kubernetes distribution because it seems like the only viable approach, remember that Kubernetes is not Linux, and consider giving Kubernetes a chance.