Autoheal | Auto restart distribute services from power cut

I have had weird problems in kubernetes. When I run install command, pods never started. Pvc was bound. It gave errors below order

I read carefully the Kubernetes Documentation here about extending the default 15% of imagefs.available and the others parameters but it doesn't say how to set it, i have installed the RKE (Rancher Kubernetes Engine) with the following configs.

I have set up a Google Cloud Compute Engine VM instance group (with number of instances between 2 and 5) and have configured autohealing to start after 3 failed health checks. The instances are created using an instance template with a startup script deploying my application. However, when I attempt to test the autohealing by stopping my application process on one vm, the failing instance is eventually removed and replaced during autohealing, but 3 new instances are also created during the process. I have also configured the instance group's autohealing initial delay also to be 600 seconds, so I don't think that is the issue.

I have checked the instance group's logs for health check statements after enabling logging, and this is what I have discovered:

1. After the first logged change in health check status, a remove instance operation is performed, followed by an add instance operation.
2. After the add instance operation, another health check probe result is logged, with health state going from "UNKNOWN"/"UNHEALTHY" to "TIMEOUT"/"UNHEALTHY".
3. Three more add instance operations are logged around 2 minutes afterwards, which are removed shortly afterwards after when scaling down.

Does anyone know why the 3 extra add instance operations are taking place and is it possible to avoid this?

I have configured a group on Google Cloud Monitoring to select gce_instances following a naming convention for a predefined instance group. However, I have noticed that it seems to include instances that have already been deleted for a brief time (ie. right after a replacement of the vms in the instance group). This is causing additional alerts to be sent for an uptime check that was created for the monitoring group because the uptime checks are still being performed for vms that are already deleted. Is there a way to configure criteria for the group to only consider vm instances that are actually running?

I have also set up autohealing for the instance group with the same triggering conditions as the uptime check which is being used in conjunction with the uptime check. Would it be possible to configure alerts on autohealing instead of using both in conjunction because of the aforementioned situation with uptime checks?

I'm attempting to stand up a new RabbitMq server (Version 3.7.23, Erlang Version 22.2.3), and I've managed to get LDAP authentication working. Unfortunately, it seems that the authentication is quite slow, so my monitoring tools sporadically report errors when checking the aliveness endpoint and the UI occasionally shows a red "could not connect to server" error when navigating through the application that eventually goes away.

While researching my issue, it seems that the rabbit_auth_backend_cache plugin should help with this. I implemented the cache but it does not seem to be working. The rabbit log still shows Rabbitmq attempting to connect to LDAP for each request and I'm not seeing any errors that would explain what's going on with the cache.

Here is my config:

I found this question..

You want to configure autohealing for network load balancing for a group of Compute Engine instances that run in multiple zones, using the fewest possible steps. You need to configure recreation of VMs if they are unresponsive after 3 attempts of 10 seconds each. What should you do?

A. Create an HTTP load balancer with a backend configuration that references an existing instance group. Set the health check to healthy(HTTP)

B. Create an HTTP load balancer with a backend configuration that references an existing instance group. Define a balancing mode and set the maximum RPS to 10.

C. Create a managed instance group. Set the Autohealing health check to healthy(HTTP)

D. Create a managed instance group. Verify that the auto scaling setting is on.

Which is the correct answer? I think is A

I'm trying to bring up a RabbitMQ cluster on Kubernetes using Rabbitmq-peer-discovery-k8s plugin and I always have only on pod running and ready but the next one always fails.

I tried multiple changes to my configuration and this is what got at least one pod running

I have set up a health check to autoheal my managed instance group. When the health check fails for a particular instance it takes a while to stop so that it can be recreated.

Is there any way to speed up the stop procedure so that my instances are recreated more quickly?

I've got some automations for setting up a clustered service [Galera] that uses etcd for service location. I've then got a load balancer that reads this information in order to initialize and run.

The trouble is that while testing autohealing I noticed that there are empty etcd 'directories' where the old nodes used to be, aka lacking a nodes subkey. This then causes trouble in the load balancer config.

Below is the result of curl http://etcd/... | jq .node.nodes[] which illustrates the problem.

How do I filter out the sub-objects that do not have a nodes key, eg: 172.17.0.16?