Backup Candidates
- Resource Configuration deployment, service definition etc.
- ETCD Cluster is where all cluster related information is stored.
- Persistent Volumes, if the applications are configured with PVC.
Resource Configuration:
The declarative approach by first creating a definition file and then running kubectl apply command on the file. This is the preferred approach if you want to save the configuration. A good practice is to store files on source code repositories. That way it can be maintained by a team the source code repository should be configured with the right backup solutions.
The imperative approach
to backing up resource configuration is to use query the Kube-apiserver. Query the Kube-apiserver using kubectl or by accessing the api server directly and save all resource configurations for all objects cretaed on the cluster has a copy.
one of the commands that can be used in a backup script is to get all pods, deployments, and services in all namespaces using the kubectl uility’s get all command and extract output
$ kubectl get all --all-namespaces -o yaml > all-deploy.yaml- VELERO can also help.
ETCD Cluster
The ETCD cluster stores information about the state of our cluster. So information about the cluster itself, the nodes and every other resource as created within the cluster are stored.
The ETCD cluster is hosted on the master nodes. While configuring ETCD we specified a location where all the data would be stored.
$ cat /etc/kubernetes/manifests/etcd.yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubeadm.kubernetes.io/etcd.advertise-client-urls: https://192.168.56.2:2379
creationTimestamp: null
labels:
component: etcd
tier: control-plane
name: etcd
namespace: kube-system
spec:
containers:
- command:
- etcd
- --advertise-client-urls=https://192.168.56.2:2379
- --cert-file=/etc/kubernetes/pki/etcd/server.crt
- --client-cert-auth=true
- --data-dir=/var/lib/etcd
- --initial-advertise-peer-urls=https://192.168.56.2:2380
- --initial-cluster=kubemaster=https://192.168.56.2:2380
- --key-file=/etc/kubernetes/pki/etcd/server.key
- --listen-client-urls=https://127.0.0.1:2379,https://192.168.56.2:2379
- --listen-metrics-urls=http://127.0.0.1:2381
- --listen-peer-urls=https://192.168.56.2:2380
- --name=kubemaster
- --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
- --peer-client-cert-auth=true
- --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
- --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
- --snapshot-count=10000
- --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
image: k8s.gcr.io/etcd:3.4.13-0
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 8
httpGet:
host: 127.0.0.1
path: /health
port: 2381
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 15
name: etcd
resources: {}
startupProbe:
failureThreshold: 24
httpGet:
host: 127.0.0.1
path: /health
port: 2381
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 15
volumeMounts:
- mountPath: /var/lib/etcd
name: etcd-data
- mountPath: /etc/kubernetes/pki/etcd
name: etcd-certs
hostNetwork: true
priorityClassName: system-node-critical
volumes:
- hostPath:
path: /etc/kubernetes/pki/etcd
type: DirectoryOrCreate
name: etcd-certs
- hostPath:
path: /var/lib/etcd
type: DirectoryOrCreate
name: etcd-data
status: {}–data-dir=/var/lib/etcd
We specified a location where all the data would be stored. The data directory. That is the directory that can be configured to be backup by the backup tool ETCD.
Also, use snapshot command
https://etcd.io/docs/v3.3.12/op-guide/recovery/
Backup by querying the Kube-apiserver. Both of these have pros and cons. If you are using a managed kubernetes environment then at times you may not even have access to the ETCD cluster. In that case, backup by querying the kube-apiserver better way.