etcd-client | An etcd v3 API client | Key Value Database library

What you have defined is only control the service account. Here's a tested spec; create a yaml file with:

For me it looks like you followed https://etcd.io/docs/v3.5/demo/#set-up-a-cluster and if this is the case, there are 2 things that you should change:

1. You used localhost in place of ${THIS_IP} variable when it should be vm IP address that is reachable from every vm in your cluster (it can be public or private address as long as it is reachable)
2. You used etcdX in --initial-cluster option and those should be IPs of vm's and not names of etcd members

What I'd recommend is setting those variables in every vm:

An etcd cluster needs a majority of nodes, a quorum, to agree on updates to the cluster state. For a cluster with n members, quorum is (n/2)+1. For any odd-sized cluster, adding one node will always increase the number of nodes necessary for quorum. Although adding a node to an odd-sized cluster appears better since there are more machines, the fault tolerance is worse since exactly the same number of nodes may fail without losing quorum but there are more nodes that can fail. If the cluster is in a state where it can’t tolerate any more failures, adding a node before removing nodes is dangerous because if the new node fails to register with the cluster (e.g., the address is misconfigured), quorum will be permanently lost.

So, in your case having two etcd nodes provide the same redundancy as one, so always recommended to have odd number of etcd nodes. code = DeadlineExceeded desc = context deadline exceeded means that client is not able to reach etcd server and it timeouts. So it might the case, that you are trying to connect to a etcd server which is down and as a results you see the error. Please refer the below doc to know more

ETDC FALUT TOLERANCE TABLE

Found a solution to this one after trying a lot of different stuff, forgot to update at the time. The certs were renewed after they had already expired, I guess this stopped a synchronisation of the certs across the different components in the cluster and nothing could talk to the API.

This is a three node cluster. I cordoned the worker nodes, stopped the kubelet service on them, stopped docker containers + service, started new docker containers, started the kubelet, uncordoned the nodes and carried out the same procedure on the master node. This forced the synchronisation of certs and keys across the different components.

The problem was allowed IPv6 on localhost. It was listening on IPv6 localhost as you can see below.

The ~/.kube/config wasn't updated with the changes.

I ran:

The issue has been resolved. After replacing certificate data in kubelet.conf as suggested in https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#check-certificate-expiration

So the problem is not exactly a bug on CRI-O as we initially thought (also the CRI-O dev team) but it seems to be a lot of configurations that need to be applied if the user desires to use CRI-O as the CRI for kubernetes and also desire to use a private repo.

So I will not put here the configurations for the CRI-O as it is already documented on the ticket that I raised with the team Kubernetes v1.18.2 with crio version 1.18.2 failing to sync with kubelet on RH7 #3915.

The first configuration that someone should apply is to configure the registries of the containers where the images will be pulled:

"etcdctl snapshot save /path/file_name.db" command works only when the api version is set to 3