filebeat | Please use the new repository

I guess the issue happened for timing out for the Kibana connection, First of all make sure that you Elasticsearch server is up and running (the default port is on 9200 ) and by typing the localhost:9200 on a browser you must get following massage

I have an update.

I am using output.console to debug the filebeats logs:

This looks like your terminal was accidentally configured for legacy ISO-646-SE or a variant. Your file is probably fine; it's just that your terminal remaps the display characters according to a scheme from the 1980s.

A quick hex dump should verify that the characters in the file are actually correct. Here's an example of what you should see.

You can use YAML's inheritance : your first input is used as a model, and the others can override parameters.

Instead of using local files, use Filebeat's Azure Module.

For this particular situation if you don't use the deafault filebeaat-* index there are issues getting the prebuilt dashboards to spin up. I dropped the custom indexing that I had in my configuration and I was able to get the dashboards to load properly.

Kubernetes has (mostly) nothing to do with this, as logging is handled by the container environment used to support Kubernetes, which is usually docker.

Depending on docker version, logs of containers could be written on json-file, journald or more, with the default being a json file. You can do a docker info | grep -i logging to check what is the Logging Driver used by docker. If the result is json-file, logs are being written down on a file in json format. If there's another value, logs are being handled in another way (and as there are various logging drivers, I suggest to check the documentation about them)

If the logs are being written on file, chances are that by using docker inspect container-id | grep -i logpath, you'll be able to see the path on the node.

Filebeat simply harvest the logs from those files and it's docker who handles the redirection between the application STDOUT inside the container and one of those files, with its driver.

Regarding exec commands not being in logs, this is an open proposal ( https://github.com/moby/moby/issues/8662 ) as not everything is redirected, just logs of the apps started by the entrypoint itself.

There's a suggested workaround which is ( https://github.com/moby/moby/issues/8662#issuecomment-277396232 )

In the mean time you can try this little hack....

echo hello > /proc/1/fd/1

Redirect your output into PID 1's (the docker container) file descriptor for STDOUT

Which works just fine but has the problem of requiring a manual redirect.

You could use a file input in read mode. Note that the default operation is to delete the file after reading it, but you can change that to just log that it was read.

filebeat can also read based on a wildcard pattern. It is lighter weight than logstash, so if you are not doing other processing in the pipeline you might prefer that.

It is possible, you'll need to look for the multiline messages in the filebeat input: https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html somthing like below would do it i think: