How to Make the Most of Kubernetes Resource Limits

Table of Contents

Kubernetes resource limits are to containerized applications what alcohol is to a party: Used in moderation, resource limits can enhance the overall performance of your applications. But if you take things to the extreme, they'll do more harm than good.

What does it mean to use Kubernetes resource limits in moderation? What's a healthy limit, and what's too low or too high? Keep reading for answers to these questions as we unpack best practices for working with Kubernetes limits.

What Are Kubernetes Resource Limits?

Kubernetes limits are the maximum amount of memory or CPU resources that a container is allowed to consume within a Kubernetes cluster.

For instance, if you want to limit a container to consuming 100 millicores of CPU and 100 megabytes of memory, you can include the following limits definitions for the container when configuring the Pod of which it is a part:

…limits:       memory: "100Mi"       cpu: "100m"…

This configuration tells Kubernetes never to allow the container to consume more than the resource limits. The container can consume fewer resources than the limits, but it can never exceed the limits.

Related:5 Key Kubernetes Trends to Follow in 2023

A Note on CPU Resource Limits in Kubernetes

Memory limits are straightforward because they are measured in megabytes, which represent the same volume of resources on any server.

In contrast, the units used to set CPU usage limits — millicores — are a little complicated. In Kubernetes, 1,000 millicores is equal to the total capacity of 1 CPU core. So, if you set a limit of 100 millicores for a container, you're telling Kubernetes to allow the container to consume only 10% of the total capacity of a CPU core.

That may seem simple. The complication comes when you consider that the computational power of a CPU core will vary depending on the type of processor of which it's a part. Plus, different cores on the same CPU could end up delivering slightly different levels of performance, due to factors such as the positioning of each core. Thus, 1,000 millicores of CPU capacity on one server, or even one particular CPU core, may be different on a different server.

The point here is that a CPU limit that allows the necessary level of performance for a container on one server might need to be changed if the container moves to a different server because the same number of millicores could translate to different levels of performance on different machines.

Related:The Late, Great K8s? 3 Reasons Why Kubernetes' Future May Be Bleak

Why Set Limits in Kubernetes?

Kubernetes doesn't require you to set limits for resources. If you don't, Kubernetes will do its best to give each container the resources it needs, when it needs them.

However, the main reason to configure limits is that they help prevent "noisy neighbor" issues. Those issues occur when one container consumes so many resources that it deprives other containers (its "neighbors") of the resources they need to run properly. You may run into these situations simply because one container ends up experiencing heavy load, or due to bugs that cause excessive CPU or memory usage.

By setting resource limits, you ensure that problematic containers can never suck up so many resources that they disrupt the performance of other containers.

In addition, resource limits are beneficial because they are a way to prioritize certain containers over others when it comes to resource allocation. You might have some workloads that are more important to you than others; for instance, some apps you are running might be for testing purposes while others are production applications. By limiting non-critical workloads to lower levels of resource consumption, you help ensure that your most important workloads always have the resources they need.

The Challenges of Kubernetes Resource Limits

While resource limits can help prevent over-consumption of resources, they do have a potential downside, too. If you set limits that are too low, you may deprive workloads of the resources they need to run optimally. Even if spare resources are available, Kubernetes won't allow containers to use them if the containers have already reached the consumption limits you set.

This means that setting overly restrictive limits could undercut Kubernetes' performance.

Limits vs. Requests

In addition to being able to set resource limits for containers, you can also (optionally) configure requests. Requests are the minimal resources that Kubernetes should make available to a given container. The container is allowed to consume more resources than those specified via a request (provided its consumption doesn't exceed any limits you've set), but setting requests helps ensure that a container has the minimum resources necessary to function properly.

You don't need to use requests if you use limits, or vice versa. But specifying both configuration variables in conjunction is a good way to establish a range of healthy resource consumption for each container. For example, if you set a request of 256 megabytes of memory and a limit of 512 megabytes, you ensure that your container will always have an amount of memory available to it that falls within the range between those two numbers.

Best Practices for Setting Kubernetes Limits

How can you know what a healthy resource limit is for each container? The following tips can help:

These practices help ensure that Kubernetes resource limits advance, rather than undercut, the performance of your Kubernetes-based workloads.