Taints and Tolerations in Kubernetes

Taints and Tolerations are used to set restrictions on what pods can be scheduled on a node.

When the pods are created Kubernetes scheduler tries to place these pods on the available worker nodes. If there are no restrictions are limitations and such the scheduler places the pods across all of the nodes to balance them out equally.

Taints are set on nodes and tolerations are set on pods.

Taint on node

$ kubectl taint nodes kubenode01 key1=value1:NoSchedule
node/kubenode01 tainted

To remove the taint with -

$ kubectl taint nodes kubenode01 key1=value1:NoSchedule-
node/kubenode01 untainted

There are three taint effects

1. NoSchedule which means the pods will not be scheduled on the node.
2. PerferNoSchedule which means the system will try to avoid placing a pod on the node but that is no guaranteed.
3. NoExecute which means that new pods will not be scheduled on the node and existing pods on the node if any will be evicted if they do not tolerate the taint.

Tolerations on a pod

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  labels:
    app: nginx
spec:
  containers:
    - name: nginx-container
      image: nginx
      ports:
        - containerPort: 80
          protocol: TCP
      
  tolerations:
    - key: "key1"
      operator: "Equal"
      value: "value1"
      effect: "NoSchedule"

Taints and tolerations are only meant to restrict nodes from accepting certain pods. It does not guarantee a pod that has the specific tolerations about the node will always be placed on the node. This means Taints and tolerations do not tell the pod to go to a particular node instead it tells the node to only accept pods with certain toleration.

When the Kubernetes cluster is the first setup it taints The master node automatically prevents any pods from being scheduled on the master node.

$ kubectl describe node kubemaster | grep Taint
Taints:             node-role.kubernetes.io/master:NoSchedule