Resource Requirements and Limits in Kubernetes

Posted on by Jay Jo

Whenever a Pod is placed on a Node, it consumes resources available to that node. It is the Kubernetes scheduler that decides which node a pod goes to. The Kubernetes scheduler takes into consideration, the amount of resources required by a pod and those available on the Nodes.

If the node has no sufficient resources, the Kubernetes scheduler avoids placing the pod on that node instead places the pod on one where sufficient resources are available if there are not sufficient resources available.

If there are not sufficient resources available on any of the nodes, Kubernetes holds back scheduling the pod, and you will see the pod in a pending state.

If you set default values for request by creating a LimitRange in the namespace (CPU request of 0.5CPU and memory of 256Mi) Kubernetes assumes that a pod or a container within a pod requires 0.5 CPU and 256 Mebibyte of memory. The is known as the resource request for a container. the minimum amount of CPU or memory requested by the container when the scheduler tries to place the pod on a node. It uses these numbers to identify a node that has a sufficient amount of resources available.

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-limit-range
spec:
  limits:
  - default:
      cpu: 1
    defaultRequest:
      cpu: 0.5
    type: Container
apiVersion: v1
kind: LimitRange
metadata:
  name: mem-limit-range
spec:
  limits:
  - default:
      memory: 512Mi
    defaultRequest:
      memory: 256Mi
    type: Container

If the application will need more than 0.5 CPU and 256 Mebibyte of memory, you can modify these values by specifying them in your pod or deployment definition files.

$ kubectl apply -f apple-pod.yaml 
pod/apple-pod created
$ kubectl get pods
NAME        READY   STATUS    RESTARTS   AGE
apple-pod   1/1     Running   0          7s
vagrant@kubemaster:~$ kubectl describe pods apple-pod
Name:         apple-pod
Namespace:    production
Priority:     0
Node:         kubenode02/192.168.56.4
Start Time:   Thu, 10 Dec 2020 12:39:36 +0000
Labels:       app=apple
              type=nginx-server
Annotations:  <none>
Status:       Running
IP:           10.36.0.2
IPs:
  IP:  10.36.0.2
Containers:
  nginx-container:
    Container ID:   docker://336f5a2c46880296999f173b0e4bf291a0681a7cacbb408d806c779ae402a0ef
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:6b1daa9462046581ac15be20277a7c75476283f969cb3a61c8725ec38d3b01c3
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 10 Dec 2020 12:39:41 +0000
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:        1
      memory:     1Gi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9w5d7 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-9w5d7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-9w5d7
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  19s   default-scheduler  Successfully assigned production/apple-pod to kubenode02
  Normal  Pulling    18s   kubelet            Pulling image "nginx"
  Normal  Pulled     15s   kubelet            Successfully pulled image "nginx" in 3.339937349s
  Normal  Created    14s   kubelet            Created container nginx-container
  Normal  Started    14s   kubelet            Started container nginx-container

You can set a limit for the resource usage on the pods.

If you set default values for limit by creating a LimitRange in the namespace (CPU request of 1CPU and memory of 512Mi). A container will be limited to consume only 1vCPU and 512 Mebibyte from the Node. 

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-limit-range
spec:
  limits:
  - default:
      cpu: 1
    defaultRequest:
      cpu: 0.5
    type: Container
apiVersion: v1
kind: LimitRange
metadata:
  name: mem-limit-range
spec:
  limits:
  - default:
      memory: 512Mi
    defaultRequest:
      memory: 256Mi
    type: Container

Also, you can change them by adding limit section under the resources.

apiVersion: v1
kind: Pod
metadata:
  name: apple-pod
  labels:
    app: apple
    type: nginx-server

spec:
  containers:
  - name: nginx-container
    image: nginx
    resources:
      requests:
        memory: "1Gi"
        cpu: 1

The requests and limits are set for each container within the pod.

apiVersion: v1
kind: Pod
metadata:
  name: apple-pod
  labels:
    app: apple
    type: nginx-server

spec:
  containers:
  - name: nginx-container
    image: nginx
    resources:
      requests:
        memory: "1Gi"
        cpu: 1
      limits:
        memory: "2Gi"
        cpu: 2  
$ kubectl apply -f apple-pod.yaml 
pod/apple-pod created 
$ kubectl describe pods apple-pod
Name:         apple-pod
Namespace:    production
Priority:     0
Node:         kubenode02/192.168.56.4
Start Time:   Thu, 10 Dec 2020 13:11:14 +0000
Labels:       app=apple
              type=nginx-server
Annotations:  <none>
Status:       Running
IP:           10.36.0.2
IPs:
  IP:  10.36.0.2
Containers:
  nginx-container:
    Container ID:   docker://3f221807c4abc2e6427439053346771098d85a1a702d51026863242afbc85522
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:6b1daa9462046581ac15be20277a7c75476283f969cb3a61c8725ec38d3b01c3
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 10 Dec 2020 13:11:20 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  2Gi
    Requests:
      cpu:        1
      memory:     1Gi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9w5d7 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-9w5d7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-9w5d7
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

when a pod tries to exceed resources beyond its specified limit,

  • CPU a container cannot use more CPU resources than its limit.
  • Memory a container can use more memory resources than its limit, so if a pod tries to consume more memory than is limit constantly, the pod will be terminated.

Resource CPU and Memory Unit

11000m1 AWS vCPU
1 GCP Core
1 Azure Core
1 Hyperthread
Resource CPU
1G (Gigabyte)1,000,000,000 bytes
1M (Megabyte)1,000,000 bytes
1K (Kilobyte)1,000 bytes
1Gi (Gibibyte)1,073,741,824 bytes
1 Mi (Mebibyte)1,048,576 bytes
1 Ki (Kibibyte)1,024 bytes
Resource Memory

Leave a Reply

Your email address will not be published.

ANOTE.DEV