8 Problems with the Kubernetes Architecture

Kubernetes, the open source container orchestrator, has become the de facto solution for anyone deploying containerized applications in production. There are many good reasons for this, including the fact that Kubernetes offers a high degree of reliability, automation and scalability. Nonetheless, I sometimes think the Kubernetes architecture is over-hyped. Despite now being more than six years old, it suffers from a variety of shortcomings. Some of them are innate to Kubernetes itself, while others are the product of the ecosystem that has grown up around the platform.

Before you go jumping on the Kubernetes bandwagon, consider the following problems with the open source container orchestrator.

1. Kubernetes is designed for web-scale companies.

First and foremost, the Kubernetes architecture is and always has been built for companies that need to manage extremely large-scale application environments.

If you’re Google (whose Borg orchestrator formed the basis for what became the open source Kubernetes project), Kubernetes is a great tool. That’s also true if you’re Netflix, Facebook, Amazon, or another web-scale company with dozens of data centers and hundreds of applications and services spread across them.

But if you’re a smaller organization that has one data center and maybe a dozen applications to deploy, Kubernetes architecture is arguably overkill. It’s like using a bulldozer to turn over the soil for a backyard garden. The effort required to configure and manage it is not worth the problems it solves unless you use it at massive scale.

This isn’t to say that Kubernetes will never be useful for smaller-scale deployments. I think it is headed in that direction. But whenever I spin up a Kubernetes cluster today to deploy just one or two applications across a small number of servers, I walk away convinced that I would have been better off using a simpler solution.

2. The Kubernetes marketplace is fractured.

Another problem with the Kubernetes architecture is that there are so many Kubernetes distributions--and so many different tools, philosophies and “opinions” associated with them--that the Kubernetes ecosystem has become highly fractured.

To a degree, of course, fracturing happens in any open source ecosystem.

For example, Red Hat Enterprise Linux has a different package manager, management tools and so on than Ubuntu Linux. However, Red Hat and Ubuntu are more similar than they are different. If you are a system admin working with Red Hat today, you don’t need to spend six months teaching yourself new tools if you want to migrate to Ubuntu.

I don’t think the same can be said of Kubernetes. If you are using, say, OpenShift today, but want to switch to VMware Tanzu, you’ll face a pretty steep learning curve. Although both of these Kubernetes distributions use the same underlying platform--Kubernetes--the methodologies and tools that they add on are wildly different.

There is a similar kind of fragmentation surrounding cloud-based Kubernetes services. Google Kubernetes Engine, or GKE, has a very different user experience and suite of management tools than does a platform like Amazon EKS, the AWS cloud’s equivalent.

This isn’t the fault of the Kubernetes architecture itself, of course. It’s the result of the way that different vendors have tried to differentiate their Kubernetes products. But it’s still a real issue from a Kubernetes user’s perspective.

3. Kubernetes has too many pieces.

We talk about Kubernetes as if it’s a single platform, but it actually consists of more than half a dozen different components. This means that when you go to install Kubernetes, or update it, you have to deal with each piece separately. And most Kubernetes distributions lack well-automated solutions for doing these things.

It’s true, of course, that Kubernetes is a complex platform, and it needs multiple parts to work. But compared to other complex platforms, Kubernetes does a particularly bad job of integrating its various parts into an easily manageable whole. Your typical Linux distribution consists of many different pieces of software, too. But you can install and manage all of them in a centralized, streamlined way. Not so with the Kubernetes architecture.

4. Kubernetes doesn’t automatically guarantee high availability.

One of the most oft-cited reasons for using Kubernetes is the notion that it magically manages your apps in such a way as to guarantee that they will never fail, even if part of your infrastructure fails.

It’s true that the Kubernetes architecture makes smart, automatic decisions about where to place workloads within a cluster. However, Kubernetes is no silver bullet for high availability. It will happily run in a production environment with just a single master node, for example, which is a recipe for bringing down your entire cluster. (If the master fails, the whole cluster will basically stop functioning.)

Kubernetes also cannot automatically guarantee that resources are properly allocated between different workloads running in a cluster. To set that up, you need to set up resource quotas manually.

5. It’s hard to control Kubernetes manually.

Despite the fact that Kubernetes requires a fair amount of manual intervention to deliver high availability, it manages to make it fairly difficult to control things manually if that’s what you actually want to do.

To be sure, there are ways to modify the timing of probes that Kubernetes performs to determine whether a container is performing properly, or to force a workload to run on a particular server within a cluster. But the Kubernetes architecture isn’t really designed with the expectation that admins will make these manual changes. It assumes that you’re always going to be happy with the defaults.

That makes sense, given that (as noted above) Kubernetes was created first and foremost for web-scale deployments. If you have thousands of servers and hundreds of workloads, you are not going to configure many things manually. But if you are a smaller shop and want to have more control over how workloads within your cluster are structured, Kubernetes does not make it easy to do that.

6. There are challenges with Kubernetes monitoring and performance optimization.

Kubernetes tries to do a good job of keeping your workloads up and running (although, as noted above, its ability to do that is contingent on factors like how many masters you set up and how you structure resource allocations).

But the Kubernetes architecture doesn’t do much to help you monitor workloads or ensure that they are performing optimally. It doesn’t alert you when there are problems, and it doesn’t really make it easy to collect monitoring data from a cluster. Nor do most of the monitoring dashboards that come with Kubernetes distributions offer in-depth visibility into your environment. There are third-party tools that give you that visibility, but those are another thing you have to set up, learn and manage if you want to run Kubernetes.

Likewise, Kubernetes is not very good at helping you optimize your costs. It doesn’t notify you if the servers in a cluster are only being used at 20% capacity, which may mean you’re wasting money on an over-provisioned infrastructure. Here again, third-party tools can help you stay on top of challenges like this, but they add more complexity.

7. Kubernetes reduces everything to code.

In Kubernetes, completing virtually any task requires you to write code. Usually, that code takes the form of YAML files, which then have to be applied on the Kubernetes command line.

Many folks would see the Kubernetes architecture’s everything-as-code requirement as a feature, not a bug. However, while I certainly understand the value of making it possible to manage the entire platform using a single methodology and tool (meaning YAML files), I do wish that Kubernetes offered other options for folks who want them.

There are times when I don’t want to write a long YAML file (or pull one from GitHub, then manually tweak random parts of it to fit my environment) to deploy a simple workload. I really wish I could just press a button or run a simple command (by which I mean a kubectl command that doesn’t require a dozen arguments, many of them configured with arcane strings of data that have to be copied and pasted) when I need to do something simple in Kubernetes. But that’s rarely the case.

8. Kubernetes wants to be all-controlling.

My final complaint about Kubernetes is that it’s really not designed to play well with other types of systems. It wants to be the only platform you use to deploy and manage your applications.

That’s good and well if all of your workloads are containerized and can be orchestrated by Kubernetes. But what if you have legacy apps that can’t run as containers? Or what if you want to run part of a workload on a Kubernetes cluster, but have another part run externally? Kubernetes doesn’t offer native functionality for doing these sorts of things. It’s designed with the assumption that everyone wants to run everything inside containers all the time.

Conclusion

Lest I be accused of hating on Kubernetes, let me restate that it is a powerful tool for orchestrating large-scale containerized applications. There are a variety of use cases for which Kubernetes is well-suited.

Yet the Kubernetes architecture also has some shortcomings. Overall, it’s not a great solution if you have legacy workloads to manage, and/or if the scale of your deployments is not large enough to justify all of the complexity that Kubernetes brings with it. To prove its full worth, Kubernetes should address these problems so that it can fully live up to the reputation that it enjoys in certain strands of the IT ecosystem.