fluent-bit | Lightweight Logs and Metrics processor

I am using AWS Opensearch to retrieve the logs from all my Kubernetes applications. I have the following pods: Kube-proxy, Fluent-bit, aws-node, aws-load-balancer-controller, and all my apps (around 10).

While fluent-bit successfully send all the logs from Kube-proxy, Fluent-bit, aws-node and aws-load-balancer-controller, none of the logs from my applications are sent. My applications had DEBUG, INFO, ERROR logs, and none are sent by fluent bit.

Here is my fluent bit configuration:

I am new to K8s. I am trying to deploy a Datadog agent to my cluster to collect logs and this happens through a ConfigMap.

First of all, I know that if it is a single $ sign that I can escape with $$ sign. My scenario is something like $log. I want that to print as it is.

Things I have tried so far:

• If I give $log, it will set an empty string
• If I give $$log, then it will set only $
• Tried with setting as a env in the same values.yaml file but same results as above

How can I resolve this issue?

My values file is like below:

Hey i have a question.

Im using logback-more-appenders(fluency plugin) to send logs to EFK stack (fluent-bit) which is working in kubernetes cluster, but it lacks kubernetes metadata ( like node/pod names).

I know i can use in logbck.xml to add Service name (because this is static), but i cannot do it to dynamic parts like node or pod name.

I tried to do it on fluent-bit side using kubernetes filter, but this works only with tail/systemd inputs not a forward one (it parses tag with filename which contains namespce and pod name). Im using forward plugin to send logs from java software to elasticsearch, and in logback.xml i cannot enter dynamic pod name (or i don't know if i can).

Any tips how i can do it? I prefer to send logs using fluency instead of sniffing host container logs.

There is an official manifest for the deployment of the daemonset https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds.yaml

Line 49 defines the volume varlibdockercontainers.

I don't understand why the Fluent-bit needs to read data from the folder /var/lib/docker/containers.

I wanted to check the following with you:

I have this kind of YAML manifest, it is a pod that is triggered when events on my app take place:

We are facing strange issue with EKS Fargate Pods. We want to push logs to cloudwatch with sidecar fluent-bit container and for that we are mounting the separately created /logs/boot and /logs/access folders on both the containers with emptyDir: {} type. But somehow the access folder is getting deleted. When we tested this setup in local docker it produced desired results and things were working fine but not when deployed in the EKS fargate. Below is our manifest files

Dockerfile

In a Docker environment my Java-App logs on STDOUT via log4j, the messages will be sent to a Graylog instance. There is no special logging config besides configuring the Console-Appender to use JsonLayout.

My docker-compose.yml snippet:

This is working as expected: