ELK-docker | Docker configuration for ELK monitoring stack | Continuous Deployment library

by sqshq Shell Version: Current License: No License

X-Ray Key Features Code Snippets Community Discussions(10)Vulnerabilities Install Support

kandi X-RAY | ELK-docker Summary

ELK-docker is a Shell library typically used in Devops, Continuous Deployment, Docker, Kafka applications. ELK-docker has no bugs, it has no vulnerabilities and it has low support. You can download it from GitHub.

Docker configuration for ELK monitoring stack with Curator and Beats data shippers support

Support

Quality

Security

License

Reuse

Support

ELK-docker has a low active ecosystem.

It has 352 star(s) with 159 fork(s). There are 31 watchers for this library.

It had no major release in the last 6 months.

ELK-docker has no issues reported. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of ELK-docker is current.

Quality

ELK-docker has no bugs reported.

Security

ELK-docker has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

License

ELK-docker does not have a standard license declared.

Check the repository for any license declaration and review the terms closely.

Without a license, all rights are reserved, and you cannot use the library in your applications.

Reuse

ELK-docker releases are not available. You will need to build from source code and install.

Installation instructions, examples and code snippets are available.

Top functions reviewed by kandi - BETA

kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of ELK-docker

Get all kandi verified functions for this library.

ELK-docker Key Features

No Key Features are available at this moment for ELK-docker.

ELK-docker Examples and Code Snippets

No Code Snippets are available at this moment for ELK-docker.

Community Discussions

Trending Discussions on ELK-docker

Logstash 7 not Adding Date Stamp to Index Name

Logging to logstash from python

Filebeat does not send logs to logstash

ELK installation on docker

How to actually bind mount a file in Docker for Windows

Running elasticsearch container on raspberry pi 3

Syslog driver not working with docker compose and elk stack

Changing ulimit value in docker run command

Can not reach Kibana remotely using ELK Docker images

Send logs to ELK container from Docker containers (without Filebeat)

QUESTION

Logstash 7 not Adding Date Stamp to Index Name

Asked 2019-Oct-25 at 15:56

I just updated from ELK 6.7.0 to 7.4.0 (via 6.8.0). After I started my 7.4.0 instance, logstash started putting the data into an index called "logstash" instead of "logstash-2019.10.24" like it used to.

I updated my output plugin to look like

...

ANSWER

Answered 2019-Oct-25 at 15:56

I finally figured this out. I had an installed template called "logstash" with the index pattern "logstash-*". This template was from a long time ago and still had the "defaults" key directly underneath "mappings". Apparently this prevented ES 7 from creating the index, so Logstash somehow fell back to the "logstash" index. I had to drop the logstash template, then recreate it without the "defaults" key. Also, I ran into some problems where records were not being indexed because of ECS - ES was assuming that certain fields (like userAgent) were one type when actually everything I have was set up to be different type. So I had to iron out those problems too.

Source https://stackoverflow.com/questions/58533409

QUESTION

Logging to logstash from python

Asked 2019-Sep-26 at 09:20

I am having some troubles logging to logstash from python.

I am using a docker image from sebp/elk (http://elk-docker.readthedocs.io) and python-logstash (https://pypi.python.org/pypi/python-logstash) as a logging handler

The docker image exposes port 5044 for the Beats interface (http://elk-docker.readthedocs.io/#usage). Whenever I try to log to this port nothing happens... Is there some other port I should be using ?

I have verified that logstash works by doing this:

...

ANSWER

Answered 2017-Jul-03 at 08:58

It does have something to do with beats, use this docker instead: github.com/deviantony/docker-elk and everything worked like a charm

Source https://stackoverflow.com/questions/44880806

QUESTION

Filebeat does not send logs to logstash

Asked 2019-May-08 at 19:40

So here's the big picture: my objective is to index large amounts of (.txt) data using the ELK stack + filebeat.

Basically, my problem is that filebeat seems to be unable to send logs to logstash. My guess is, some docker networking config is off...

The code for my project is available at https://github.com/mhyousefi/elk-docker.

THE ELK CONTAINER

To do so, I have one docker-compose.yml to run a container from the image sebp/elk, which looks like this:

...

ANSWER

Answered 2018-Aug-15 at 10:21

Networking in namespaced in containers by default, which means each container gets it's own private ip, and localhost in a container is local to just that container.

That means you need to specify the DNS entry of the elastic server, not localhost in your config file. With compose and swarm mode, the service name is automatically setup with a DNS entry pointing to your containers:

Source https://stackoverflow.com/questions/51849542

QUESTION

ELK installation on docker

Asked 2018-Oct-29 at 14:18

I took ELK container from here using the command :

...

ANSWER

Answered 2018-Sep-19 at 00:21

There is an open issue on github for this error, https://github.com/elastic/ansible-elasticsearch/issues/430

elasticsearch tries to write and read from its config directory in version 6.x. Current fix is to explicitly set permission to g+ws, i.e.

Source https://stackoverflow.com/questions/52221774

QUESTION

How to actually bind mount a file in Docker for Windows

Asked 2018-Apr-28 at 22:17

I'm attempting to update the sebp/elk Logstash configuration following the documentation here. I'm running into a situation in which the host file that I am attempting to mount is being mounted as a directory in the container.

I found this related question How to mount a single file in a volume but the notion of running with PWD didn't work for me on Windows as I got the following error: C:\Program Files\Docker\Docker\Resources\bin\docker.exe: invalid reference format

I'm running Docker on Windows 10 (Build 16299.192)

...

ANSWER

Answered 2018-Apr-28 at 22:17

This works for me:

Note: This approach addresses W10 Home Edition with Docker Toolbox and VirtualBox.

Overview: Create a folder in local-machine, mount this as a shared folder in Docker VM, use this shared folder as a bindmount to Docker container.

Stop docker VM using docker-machine stop default
Open VirtualBox, find default go to Settings > Shared Folder
You will see c/Users is binded to your c:\Users
Add a new shared folder, note the name it is assigned. Let's name this as [local-shared]
Exit Settings
docker-machine start default
Once started, docker-machine ssh default
sudo vi /mnt/sda1/var/lib/boot2docker/profile
Append the following:

Source https://stackoverflow.com/questions/48159422

QUESTION

Running elasticsearch container on raspberry pi 3

Asked 2017-Oct-05 at 08:19

I am trying to run an containerized Elasticsearch of ELK stack on rpi3. There is no existing image on docker hub, so I am trying to build one from scratch. I used Dockerfile as base and change the base image to resin/rpi-raspbian. However, I encountered several problems:

For ES 5.4.0+, ES process failed to start up due to missing libjnidispatch.so in elastic-bundled jar file (Error: Native library (com/sun/jna/linux-arm/libjnidispatch.so) not found in resource path)
Therefore, I downgrade to se v5.3.2 instead. I got around jna problem and but ES still failed to bootstrap because due to runtime exception (seccomp unavailable: 'arm' architecture unsupported)

Elasticsearch log follows below. [2017-10-03T13:05:44,366][WARN ][o.e.b.JNANatives ] unable to install syscall filter: java.lang.UnsupportedOperationException: seccomp unavailable: 'arm' architecture unsupported

P.S. I already reduce heap size to 512m.

Is there anything I should do to make this happen? Thanks.

...

ANSWER

Answered 2017-Oct-05 at 08:18

Answering my own question in case someone also stumble on the same problem. However, any other answers are welcome!

I came across with a git repo that builds a docker image running on rpi boards. It seems the author has modified the original elasticsearch source code which adds seccomp support on ARM. The docker repo for the modified distribution can be found here.

I have ElasticSearch 2.4 successfullly running on my raspberry pi 3 board. Here is my git repo for reference

Source https://stackoverflow.com/questions/46543198

QUESTION

Syslog driver not working with docker compose and elk stack

Asked 2017-Sep-25 at 14:21

I want to send logs from one container running my_service to another running the ELK stack with the syslog driver (so I will need the logstash-input-syslog plugin installed).

I am tweaking this elk image (and tagging it as elk-custom) via the following Dockerfile-elk

(using port 514 because this seems to be the default port)

...

ANSWER

Answered 2017-Sep-25 at 14:21

You need to use tcp://127.0.0.1:514 instead of tcp://elk-custom:514. Reason being this address is being used by docker and not by the container. That is why elk-custom is not reachable.

So this will only work when you map the port (which you have done) and the elk-service is started first (which you have done) and the IP is reachable from the docker host, for which you would use tcp://127.0.0.1:514

Source https://stackoverflow.com/questions/46406680

QUESTION

Changing ulimit value in docker run command

Asked 2017-Jul-18 at 11:30

I tried to install this image...

...

ANSWER

Answered 2017-Jul-18 at 11:30

Use the form: docker run --ulimit =:

So, for your nofile, an example would be --ulimit nofile=65536:65536

Docs:

https://docs.docker.com/engine/reference/commandline/run/#set-ulimits-in-container-ulimit

Source https://stackoverflow.com/questions/45165178

QUESTION

Can not reach Kibana remotely using ELK Docker images

Asked 2017-Apr-21 at 15:47

I have a remote Ubuntu 14.04 machine. I downloaded and ran a couple of ELK Docker images, but I seem to be getting the same behavior in all of them. I tried the images in these two repositories: spujadas/elk-docker and deviantony/docker-elk. The problem is, in both images, Elasticsearch, Logstash and Kibana all work perfectly locally, however when I try to reach Kibana from a remote computer using http://host-ip:5601, I get a connection timeout and can't reach Kibana. Also, I can reach Elasticsearch from http://host-ip:9200. As both the repositories suggest, I injected some data into Logstash, but that didn't work either. Is there some tweak I need to make in order to reach Kibana remotely?

EDIT: I tried opening up port 5601 as suggested here, but that didn't work either.

...

ANSWER

Answered 2017-Apr-21 at 15:47

As @Rawkode suggested in the comments, the problem was the firewall. The VM I'm working on was created on Azure and I had to create an inbound security rule to allow Kibana to be accessed from port 5601. More on this subject can be read from here.

Source https://stackoverflow.com/questions/43497064

QUESTION

Send logs to ELK container from Docker containers (without Filebeat)

Asked 2017-Jan-07 at 23:20

I'm using the SEBP/ELK Docker container as it appears to be the most suitable for quickly setting up application logging. Unfortunately, after reading through the docs there doesn't seem to be an easy way to get logs from sibling containers without the use of Filebeat.

I don't want to install Filebeat on each of my containers because that seems like it goes directly against Docker's separation of duties mantra.

TLDR; how do I get logs from my application containers to my ELK container?

...

ANSWER

Answered 2017-Jan-06 at 07:18

Using filebeat in each container is against Docker's philosophy. It will be waste of resources, And have more management overhead.

You can use local log file via logstash.

Example config:

Source https://stackoverflow.com/questions/41498146

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install ELK-docker

Install required Beats shippers on the host which should be monitored
Install Docker and Docker Compose on the ELK host
Clone this repository and hit docker-compose build

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: