graylog2-server | Free and open source log management

by Graylog2 Java Version: Current License: Non-SPDX

Download this library from

kandi X-RAY | graylog2-server Summary

graylog2-server is a Java library typically used in Logging, Kafka applications. graylog2-server has build file available and it has medium support. However graylog2-server has 100 bugs, it has 3 vulnerabilities and it has a Non-SPDX License. You can download it from GitHub, Maven.

Free and open source log management

Support

Quality

Security

License

Reuse

kandi-support Support

graylog2-server has a medium active ecosystem.
It has 6088 star(s) with 959 fork(s). There are 249 watchers for this library.
It had no major release in the last 12 months.
There are 1320 open issues and 4521 have been closed. On average issues are closed in 212 days. There are 95 open pull requests and 0 closed requests.
It has a neutral sentiment in the developer community.
The latest version of graylog2-server is current.

graylog2-server Support

Best in #Java

Average in #Java

graylog2-server Support

Best in #Java

Average in #Java

quality kandi Quality

graylog2-server has 100 bugs (5 blocker, 1 critical, 19 major, 75 minor) and 1488 code smells.

graylog2-server Quality

Best in #Java

Average in #Java

graylog2-server Quality

Best in #Java

Average in #Java

security Security

graylog2-server has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
graylog2-server code analysis shows 3 unresolved vulnerabilities (1 blocker, 2 critical, 0 major, 0 minor).
There are 12 security hotspots that need review.

graylog2-server Security

Best in #Java

Average in #Java

graylog2-server Security

Best in #Java

Average in #Java

license License

graylog2-server has a Non-SPDX License.
Non-SPDX licenses can be open source with a non SPDX compliant license, or non open source licenses, and you need to review them closely before use.

graylog2-server License

Best in #Java

Average in #Java

graylog2-server License

Best in #Java

Average in #Java

build Reuse

graylog2-server releases are not available. You will need to build from source code and install.
Deployable package is available in Maven.
Build file is available. You can build the component from source.
graylog2-server saves you 22902 person hours of effort in developing the same functionality from scratch.
It has 44831 lines of code, 3791 functions and 711 files.
It has medium code complexity. Code complexity directly impacts maintainability of the code.

graylog2-server Reuse

Best in #Java

Average in #Java

graylog2-server Reuse

Best in #Java

Average in #Java

Top functions reviewed by kandi - BETA

kandi has reviewed graylog2-server and discovered the below as its top functions. This is intended to give you an instant insight into graylog2-server implemented functionality, and help decide if they suit your requirements.

Extracts all values from a PivotResult .
Iterates over the REST resources and returns a map of the model classes .
Walk the expression tree .
Returns mongo stats .
Decode a frame .
Launch legacy checkpoint .
Generate the aggregate query part .
Upgrade the elasticsearch configuration .
Reads from the journal .
Post processing the message .

graylog2-server Key Features

Free and open source log management

graylog2-server Examples and Code Snippets

No Code Snippets are available at this moment for graylog2-server.

See all Code Snippets related to Java

Community Discussions

Trending Discussions on Logging

How to dynamically pass variable to class __init__ method in a custom python logging class
MySql 5.5; possible to exclude a table from logging?
Exclude Logs from Datadog Ingestion
How can I add CSV logging mechanism in case of Multivariable Linear Regression using TensorFlow?
Implementing an interface with a generic parameter on a F# record
How to get message from logging function?
Python disabled logging slowing script
Xcode log "Writing analzed variants"
Was slf4j affected with vulnerability issue in log4j
Simple Logger erroring on init

Trending Discussions on Logging

QUESTION

How to dynamically pass variable to class __init__ method in a custom python logging class

Asked 2022-Apr-01 at 15:20

It is difficult to put what I need in a sentence but the code below pretty much explains it:

I have my logging class in a separate file (log_file) as below, and a logger object defined there:

from io import StringIO
import logging

class NetddLog():
   def __init__(self, name, format="%(asctime)s %(levelname)s %(message)s", level=logging.INFO):
      self.name = name
      self.level = level
      self.format = format

      #Logger configuration.
      self.formatter = logging.Formatter(self.format)
      self.logger = logging.getLogger(name)#name
      self.logger.setLevel(self.level)
      #Logging to memory
      self.log_capture_string = StringIO()
      self.ch_logger = logging.StreamHandler(self.log_capture_string)
      self.ch_logger.setFormatter(self.formatter)
      self.logger.addHandler(self.ch_logger)

   def debug(self, msg, extra=None):
      self.logger.debug(msg, extra=extra)

ip_logger = NetddLog("IP_LOG")

In another file (ip_file), I have my ping function as below:

from log_file import ip_logger
from icmplib import ping

def ping_ip(ip_num, ip):
    try: 
        ip_logger.info(f"{ip_num}: Pinging {ip} started")

        host = ping(ip, count=4, interval=1, timeout=2, payload_size=64, privileged=True)
        if host.is_alive:
            ip_logger.info(f"{ip_num}: Pinging {ip} succeded")
        else:
            raise Exception 

    except Exception as err:
        ip_logger.error(f"{ip_num}: Pinging {ip} failed {err}")

The ip_num is the number of the IP address(ip) in a IP addr list, in another file (main_file), from which I call ping_ip(ip_num, ip)

The log messages prints just fine, but I'm putting ip_num inside the actual log message, each time. What I want to do is, having included in the format of the logger when the logger is created in the class, and probably just call the function with ping_ip(ip)

So the format in the init class method will look something like this: format=f"%(asctime)s %(levelname)s {ip_num}: %(message)s" , this way I don't have to include ip_num inside every log message I create. Is there a way to achieve this within the current class configuration or an alternative way? (I want to keep things separated as much as possible and not put everything in the main_file)

Update: Based on a previous answer I just redefined the logging methods inside the class to add an extra parameter in the format. For instance info function would change like below, and %(ip_num)s can be added to the format.

   def info(self, ip_num, msg):
      self.d = {'ip_num': f"{ip_num}"}
      self.logger.info(msg, extra=self.d)

ANSWER

Answered 2022-Mar-31 at 21:12

Yes, you can achieve what you want, actually is well documented under: https://docs.python.org/3/howto/logging.html

There is a parameter where you can provide a dictionary with additional values to your log format.

Below you can find the snippet which does the job:

import logging


def config_log(FORMAT = '%(asctime)s %(levelname)s IP:%(ip_num)s %(message)s'):
    logging.basicConfig(filename='example.log', encoding='utf-8',format=FORMAT, level=logging.INFO)
    
def log_something(ipnum, mymessage):
    d = {'ip_num': f"{ipnum}"}
    logging.info(mymessage, extra=d)

if __name__ == "__main__":       
    config_log() 
    log_something("127.0.0.1",'Here is your message log')

Source https://stackoverflow.com/questions/71698579

QUESTION

Exclude Logs from Datadog Ingestion

Asked 2022-Mar-19 at 22:38

I have a kubernetes cluster that's running datadog and some microservices. Each microservice makes healthchecks every 5 seconds to make sure the service is up and running. I want to exclude these healthcheck logs from being ingested into Datadog.

I think I need to use log_processing_rules and I've tried that but the healthcheck logs are still making it into the logs section of Datadog. My current Deployment looks like this:

apiVersion: apps/v1
kind: Deployment
[ ... SNIP ... ]
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-service
  template:
    metadata:
      labels:
        app: my-service
        version: "fac8fb13"
      annotations:
        rollme: "IO2ad"
        tags.datadoghq.com/env: development
        tags.datadoghq.com/version: "fac8fb13"
        tags.datadoghq.com/service: my-service
        tags.datadoghq.com/my-service.logs: |
          [{
            "source": my-service,
            "service": my-service,
            "log_processing_rules": [
              {
                "type": "exclude_at_match",
                "name": "exclude_healthcheck_logs",
                "pattern": "\"RequestPath\": \"\/health\""
              }
            ]
          }]

and the logs coming out of the kubernetes pod:

$ kubectl logs my-service-pod

{
  "@t": "2022-01-07T19:13:05.3134483Z",
  "@m": "Request finished HTTP/1.1 GET http://10.64.0.80:5000/health - - - 200 - text/plain 7.5992ms",
  "@i": "REDACTED",
  "ElapsedMilliseconds": 7.5992,
  "StatusCode": 200,
  "ContentType": "text/plain",
  "ContentLength": null,
  "Protocol": "HTTP/1.1",
  "Method": "GET",
  "Scheme": "http",
  "Host": "10.64.0.80:5000",
  "PathBase": "",
  "Path": "/health",
  "QueryString": "",
  "HostingRequestFinishedLog": "Request finished HTTP/1.1 GET http://10.64.0.80:5000/health - - - 200 - text/plain 7.5992ms",
  "EventId": {
    "Id": 2,
    "Name": "RequestFinished"
  },
  "SourceContext": "Microsoft.AspNetCore.Hosting.Diagnostics",
  "RequestId": "REDACTED",
  "RequestPath": "/health",
  "ConnectionId": "REDACTED",
  "dd_service": "my-service",
  "dd_version": "54aae2b5",
  "dd_env": "development",
  "dd_trace_id": "REDACTED",
  "dd_span_id": "REDACTED"
}

EDIT: Removed 2nd element of the log_processing_rules array above as I've tried with 1 and 2 elements in the rules array.

EDIT2: I've also tried changing log_processing_rules type to INCLUDE at match in an attempt to figure this out:

"log_processing_rules": [
  {
    "type": "include_at_match",
    "name": "testing_include_at_match",
    "pattern": "somepath"
  }
]

and I'm still getting the health logs in Datadog (in theory I should not as /health is not part of the matching pattern)

ANSWER

Answered 2022-Jan-12 at 20:28

I think the problem is that you're defining multiple patterns; the docs state, If you want to match one or more patterns you must define them in a single expression.

Try somtething like this and see what happens:

"log_processing_rules": [
  {
    "type": "exclude_at_match",
    "name": "exclude_healthcheck_logs",
    "pattern": "\/health|\"RequestPath\": \"\/health\""
  }

Source https://stackoverflow.com/questions/70687054

QUESTION

How can I add CSV logging mechanism in case of Multivariable Linear Regression using TensorFlow?

Asked 2022-Feb-04 at 07:28

Suppose, the following is my Multivariable Linear Regression source code in Python:

import os

os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"

import sys, random
import time

import tensorflow as tf
from   tensorflow import keras
from   tensorflow.keras.models import Sequential
from   tensorflow.keras.layers import Dense
from   tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt
import numpy as np

def load_data_k(fname: str, yyy_index: int, **selection):
    i = 0
    file = open(fname)
    if "top_n_lines" in selection:
        lines = [next(file) for _ in range(int(selection["top_n_lines"]))]
    elif "random_n_lines" in selection:
        tmp_lines = file.readlines()
        lines = random.sample(tmp_lines, int(selection["random_n_lines"]))
    else:
        lines = file.readlines()

    data_x, data_y = [], []
    for l in lines:
        row = l.strip().split()   
        x = [float(ix) for ix in row[yyy_index+1:]] 
        y = float(row[yyy_index])  
        data_x.append(x) 
        data_y.append(y) 
    # END for l in lines...

    num_rows = len(data_x)

    print("row size = ", len(data_x[0]))

    given_fraction = selection.get("validation_part", 1.0)
    if given_fraction > 0.9999:
        valid_x, valid_y = data_x, data_y
    else:
        n = int(num_rows * given_fraction)
        data_x, data_y = data_x[n:], data_y[n:]
        valid_x, valid_y = data_x[:n], data_y[:n]
    # END of if-else block

    print("size of x = ", len(data_x))
    print("size of y = ", len(data_y))

    tx = tf.convert_to_tensor(data_x, dtype=tf.float32)
    ty = tf.convert_to_tensor(data_y, dtype=tf.float32)

    vx = tf.convert_to_tensor(valid_x, dtype=tf.float32)
    vy = tf.convert_to_tensor(valid_y, dtype=tf.float32)

    return tx, ty, vx, vy
# END of the function


# load training data from the disk
train_x, train_y, validate_x, validate_y = \
        load_data_k(
            fname="data_file.csv",
            yyy_index=6,
            random_n_lines=90000,
            validation_part=0.2
        )

print("training data size : ", len(train_x))
print("validation data size : ", len(validate_x))

predict_data = np.array([[7.042, 5.781, 5.399, 5.373, 5.423, -9.118, 5.488,   5.166,   4.852,  7.470,  6.452,  6.069,
                         0,     0,     0,     1,     0,      1,     1,       3,       0,      0,      0,      1, 
                         0,     0,     0,     0,     0,      0,     0,       0,       0,      0,      0,      0, 
                         0,     0,     0,     0]])


# Create Keras model
model = Sequential()
model.add(Dense(1, input_dim=40))
model.add(Dense(128))
model.add(Dense(128))
model.add(Dense(1))

# Gradient descent algorithm
adam_opt = Adam(0.1)

model.compile(loss='mse', optimizer=adam_opt)

history = model.fit(train_x, train_y, epochs=500)

prediction = model.predict(predict_data)
print(prediction)

I want to add CSV logging for the training loss, validation loss, training accuracy, and validation accuracy.

How can I do that?

ANSWER

Answered 2022-Feb-04 at 07:28

Just use the tf.keras.callbacks.CSVLogger and any regression metric you want to log during training:

import tensorflow as tf

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(1, input_dim=40))
model.add(tf.keras.layers.Dense(128))
model.add(tf.keras.layers.Dense(128))
model.add(tf.keras.layers.Dense(1))

adam_opt = tf.keras.optimizers.Adam(0.1)
model.compile(loss='mse', optimizer=adam_opt, metrics=tf.keras.metrics.MeanSquaredError(name="mean_squared_error", dtype=None))

train_x = tf.random.normal((50, 40))
train_y = tf.random.normal((50, 1))
val_x = tf.random.normal((50, 40))
val_y = tf.random.normal((50, 1))

csv_logger = tf.keras.callbacks.CSVLogger('model_training.csv')
history = model.fit(train_x, train_y, epochs=5, validation_data=(val_x, val_y), callbacks=[csv_logger])

model_training.csv:

epoch        loss  mean_squared_error    val_loss  val_mean_squared_error
    0  304.349060          304.349060   69.584991               69.584991
    1  105.304787          105.304787  170.063126              170.063126
    2  175.232788          175.232788    7.874812                7.874812
    3  104.159607          104.159607  320.626556              320.626556
    4  194.709763          194.709763    1.438866                1.438866

Source https://stackoverflow.com/questions/70910193

QUESTION

Implementing an interface with a generic parameter on a F# record

Asked 2022-Jan-28 at 03:51

I'm trying to implement the an Microsoft.Extensions.Logging.ILogger (copied below for brevity) on a F# Record

using System;

namespace Microsoft.Extensions.Logging
{
    public interface ILogger
    {
        void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter);
        bool IsEnabled(LogLevel logLevel);
        IDisposable BeginScope<TState>(TState state);
    }
}

This is the record implementation.

  type ILoggerRcd<'TState> = 
    {
      BeginScope : 'TState -> IDisposable
      IsEnabled : LogLevel -> bool
      Log : LogLevel * EventId * 'TState * exn * Func<'TState,exn,string> -> unit
    }
    interface ILogger with
          override this.BeginScope(state : 'TState): IDisposable = 
              this.BeginScope (state)
          override this.IsEnabled(logLevel: LogLevel): bool = 
              this.IsEnabled logLevel
          override this.Log(logLevel: LogLevel, eventId: EventId, state : 'TState, ``exception``: exn, formatter: Func<'TState,exn,string>): unit = 
              this.Log(logLevel, eventId, state, ``exception``, formatter)

However, I'm getting this error on BeginScope: One or more of the explicit class or function type variables for this binding could not be generalized, because they were constrained to other types.

And this error on Log: The generic member 'Log' has been used at a non-uniform instantiation prior to this program point. Consider reordering the members so this member occurs first. Alternatively, specify the full type of the member explicitly, including argument types, return type and any additional generic parameters and constraints.

I've read through a few issues on fsharp compiler itself but nothing seems to be exactly this case I'm hitting. Is this something that can be done?

ANSWER

Answered 2022-Jan-28 at 03:34

The ILogger interface requires that you can log objects of any type, but you're trying to log only those of type 'TState.

Take the signature of BeginScope:

IDisposable BeginScope<TState>(TState state);

See that <TState> bit there? That's a generic parameter. This signature means that every time anybody calls this method, they get to choose a type TState to use with that call.

Again: the caller chooses the generic type, not the implementer.

For example:

let l : ILogger = ...
l.BeginScope 42   // TState = int
l.BeginScope true // TState = bool

This means that your implementation of BeginScope has to be able to work with any type, not just the type with which your ILoggerRcd record was created.

Source https://stackoverflow.com/questions/70886932

QUESTION

How to get message from logging function?

Asked 2022-Jan-18 at 13:27

I have a logger function from logging package that after I call it, I can send the message through logging level.

I would like to send this message also to another function, which is a Telegram function called SendTelegramMsg().

How can I get the message after I call the funcion setup_logger send a message through logger.info("Start") for example, and then send this exatcly same message to SendTelegramMsg() function which is inside setup_logger function?

My currently setup_logger function:

# Define the logging level and the file name
def setup_logger(telegram_integration=False):
    """To setup as many loggers as you want"""

    filename = os.path.join(os.path.sep, pathlib.Path(__file__).parent.resolve(), 'logs', str(dt.date.today()) + '.log')
    formatter = logging.Formatter('%(levelname)s: %(asctime)s: %(message)s', datefmt='%m/%d/%Y %H:%M:%S')
    level = logging.DEBUG

    handler = logging.FileHandler(filename, 'a')    
    handler.setFormatter(formatter)

    consolehandler = logging.StreamHandler()
    consolehandler.setFormatter(formatter)

    logger = logging.getLogger('logs')
    if logger.hasHandlers():
        # Logger is already configured, remove all handlers
        logger.handlers = []
    else:
        logger.setLevel(level)
        logger.addHandler(handler)        
        logger.addHandler(consolehandler)

    #if telegram_integration == True:
        #SendTelegramMsg(message goes here)

    return logger

After I call the function setup_logger():

logger = setup_logger()
logger.info("Start")

The output:

INFO: 01/06/2022 11:07:12: Start

How am I able to get this message and send to SendTelegramMsg() if I enable the integration to True?

ANSWER

Answered 2022-Jan-06 at 15:59

Picking up the idea suggested by @gold_cy: You implement a custom logging.Handler. Some hints for that:

for the handler to be able to send message via a bot, you may want to pass the bot to the handlers __init__ so that you have it available later
emit must be implemented by you. Here you'll want to call format which gives you a formatted version of the log record. You can then use that message to send it via the bot
Maybe having a look at the implementation of StreamHandler and FileHandler is helpful as well

Source https://stackoverflow.com/questions/70608619

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install graylog2-server

You can download it from GitHub, Maven.
You can use graylog2-server like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the graylog2-server component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

Help us build the future of log management and be part of a project that is used by thousands of people out there every day. Follow the contributors guide and read the contributing instructions to get started.