fluentd-kubernetes-daemonset | Fluentd daemonset for Kubernetes and it Docker image

I'm using AWS EKS 1.21 with Fargate (serverless). I'm trying to run Fluentd as a daemonset however the daemonset is not running at all.

All the other objects like role, rolebinding, serviceaccount, configmap are already in place in the cluster.

We're using https://github.com/fluent/fluentd-kubernetes-daemonset to deploy Fluentd in our K8s cluster. We have 5 nodes in the cluster, which means there are 5 Fluentd pods.

Each Fluentd pod in the DaemonSet exposes Prometheus metrics on localhost:24231/metrics endpoint via fluentd prometheus plugin. I'm having trouble finding the relevant bits of documentation on how to configure Prometheus to collect those metrics from every Pod's localhost:24321/metrics endpoint.

• there are N pods in a DaemonSet
• each pod has prometheus metrics exposed on localhost:24321

I need to configure Prometheus so it's able to scrape those metrics somehow. Any tips on how to solve this or examples of such configurations would be much appreciated!

I have to run two CronJobs in Kubernetes (AWS-EKS) and I have below configuration. When I apply the template, only one CronJob is getting created. The one that gets created is always the second one. So it looks like the first one is getting overwritten by the second. I am unable to figure out what am I doing wrong.

EDIT : I harcoded the fluentd service IP directly in my express app and its working.. how to get it work without harcoding ip?

I have a couple of pods (nodejs + express server) running on a Kubernetes cluster.

I'd like send logs from my nodejs pods to a Fluentd DeamonSet.

But I'm getting this error :

Fluentd error Error: connect ECONNREFUSED 127.0.0.1:24224

I'm using https://github.com/fluent/fluent-logger-node and my configuration is pretty simple:

I'm using that fluentd daemonset docker image and sending logs to ES with fluentd is working perfectly by the way of using following code-snippets:

I got the fluentd-kubernetes-daemonset charts from https://github.com/fluent/fluentd-kubernetes-daemonset, and deployed fluentd into kube-system namespace as daemonset. It sends entire cluster logs to elasticsearch. We deploy our csc application in the csc namespace. Instead of installing fluentd as daemonset to collect entire cluster logs, we would like to deploy fluentd in the csc namespace only, and only send csc logs (logs in csc namespace) to elasticsearch. Is there a way to do it?

We use AWS Elasticsearch Service. We would like to install fluentd to our Kubernetes cluster. Fluentd sends logs to AWS Elasticsearch Service. It there any instructions on how to do it? At this link, https://docs.fluentd.org/v/0.12/articles/kubernetes-fluentd, it shows steps on sending the logs to an Elasticsearch Pod. It means the elasticsearch is installed inside the Kubernetes cluster.

The link above shows make changes in fluentd-daemonset-elasticsearch.yaml (see below) for elasticsearch pod. What configuration changes I need to make for the AWS elasticsearch service?

EDIT: I've added new information per my comment in response to efrat-levitan's suggestion. The log output listed is slightly different as I upgraded my Elasticsearch version to 7.6.0 as suggested by the original comment. To help debugging, I also didn't start Elasticsearch right away. The effect of this can be seen in the ECONNREFUSED messages in the log. I called out the log file changes in the summary below. Most of the rest of the message text (i.e., not log snippets) remains the same as before.

I've been working on getting an ARM version (for a Raspberry Pi 3 & 4) of fluentd with the fluent-plugin-elasticsearch plugin running in docker. I haven't been able to find an appropriate docker image so I've built one on my own (if anyone knows where I can find one I'd appreciate it). I started with the fluentd-docker-image repo (doesn't include Elasticsearch plugins) and modified it as I thought necessary using the fluentd-kubernetes-daemonset repo (does include the Elasticsearch plugins). The good news is that it starts up just fine on a Raspberry Pi. The bad news is that it appears like it doesn't even try to connect to ElasticSearch (external to the Raspberry Pi network. The log file looks like this:

I have a local kubernetes cluster where I added a Fluentd Daemonset using the preconfigured elasticsearch image (fluent/fluentd-kubernetes-daemonset:elasticsearch). Step 2 of this article. I also have an elastic cluster running in the cloud. You can pass some env variables to the fluentd-elasticsearch image for configuration. It looks pretty straightforward, but when running the fluentd Pod I keep getting the error:

"Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"fa0acce34bf64db9bc9e46f98743c185.westeurope.azure.elastic-cloud.com\", :port=>9243, :scheme=>\"https\", :user=>\"username\", :password=>\"obfuscated\"})!" plugin_id="out_es"

when I try to reach the elastic cluster from within the pod with # wget https://fa0acce34bf64db9bc9e46f98743c185.westeurope.azure.elastic-cloud.com:9243/ I get a 401 unauthorized (cuz I havent submitted user/pass here), but it at least shows that the address is reachable.

Why is it failing to connect? I already set the FLUENT_ELASTICSEARCH_SSL_VERSION to 'TLSv1_2', i saw that that solved some problems for others.

Daemonset configuration: