coredns | CoreDNS is a DNS server that chains plugins | DNS library

We have a 2 node K3S cluster with one master and one worker node and would like "reasonable availability" in that, if one or the other nodes goes down the cluster still works i.e. ingress reaches the services and pods which we have replicated across both nodes. We have an external load balancer (F5) which does active health checks on each node and only sends traffic to up nodes.

Unfortunately, if the master goes down the worker will not serve any traffic (ingress).

This is strange because all the service pods (which ingress feeds) on the worker node are running.

We suspect the reason is that key services such as the traefik ingress controller and coredns are only running on the master.

Indeed when we simulated a master failure, restoring it from a backup, none of the pods on the worker could do any DNS resolution. Only a reboot of the worker solved this.

We've tried to increase the number of replicas of the traefik and coredns deployment which helps a bit BUT:

• This gets lost on the next reboot
• The worker still functions when the master is down but every 2nd ingress request fails
  • It seems the worker still blindly (round-robin) sends traffic to a non-existant master

We would appreciate some advice and explanation:

• Should not key services such as traefik and coredns be DaemonSets by default?
• How can we change the service description (e.g. replica count) in a persistent way that does not get lost
• How can we get intelligent traffic routing with ingress to only "up" nodes
• Would it make sense to make this a 2-master cluster?

UPDATE: Ingress Description:

After I deployed the webui (k8s dashboard), I logined to the dashboard but nothing found there, instead a list of errors in notification.

I'm following a tutorial https://docs.openfaas.com/tutorials/first-python-function/,

currently, I have the right image

I'm trying to deploy a simple REST API written in Golang to AWS EKS.

I created an EKS cluster on AWS using Terraform and applied the AWS load balancer controller Helm chart to it.

All resources in the cluster look like:

I have a cluster in AWS created by these instructions.

Then I tried to add nodes in this cluster according to this documentation.

It seems that the nodes fail to be created with vpc-cni and coredns health issue type: insufficientNumberOfReplicas The add-on is unhealthy because it doesn't have the desired number of replicas.

The status of the pods kubectl get pods -n kube-system:

I faced this problem since yesterday, no problems before.
My environment is

• Windows 11
• Docker Desktop 4.4.4
• minikube 1.25.1
• kubernetes-cli 1.23.3

I'm trying to set up FluentBit for my EKS cluster in Terraform, via this module, and I have couple of questions:

cluster_identity_oidc_issuer - what is this? Frankly, I was just told to set this up, so I have very little knowledge about FluentBit, but I assume this "issuer" provides an identity with needed permissions. For example, Okta? We use Okta, so what would I use as a value in here?

cluster_identity_oidc_issuer_arn - no idea what this value is supposed to be.

worker_iam_role_name - as in the role with autoscaling capabilities (oidc)?

This is what eks.tf looks like:

I've googled few days and haven't found any decisions. I've tried to update k8s from 1.19.0 to 1.19.6 In Ubuntu-20. (cluster manually installed k81 - master and k82 - worker node)

Something wrong happend with my RPi 4 cluster based on k3sup.

Everything works as expected until yesterday when I had to reinstall master node operating system. For example, I have a redis installed on master node and then some pods on worker nodes. My pods can not connect to redis via DNS: redis-master.database.svc.cluster.local (but they do day before).

It throws an error that can not resolve domain when I test with busybox like: