Plog | Plog 是 Parse Log 的缩写

 by   CNSRE Python Version: Current License: No License

kandi X-RAY | Plog Summary

kandi X-RAY | Plog Summary

Plog is a Python library. Plog has no bugs, it has no vulnerabilities and it has low support. However Plog build file is not available. You can download it from GitHub.

Plog 是 "Parse Log" 的缩写,是一套处理日志流的框架,日志流格式可以是Apache,nginx等常规意义的日志格式,也可以是自定义格式. 受[FlumeNG] ,sink,已经完成了主体的共有的可以抽象出来的功能,比如线程的同步互斥,消息的生产消费,处理时间间隔的控制,还有一些简单的source,channel and sink函数. #定义解析数据的模块名 channel_module=regrex_channel #如是正则,定义正则规则 channel_filter_regex=([\w\d.\s,]{0,})\s()\s(?P\d|-)\s(\w+)\s\[([\[\]])\s\\d+\]\s"((?:["]|\"))"\s(?P\d{3})\s(\d|-)\s"((?:["]|\")|-)"\s"(.|-)"\s"((?:["]|\"))"\s"(.|-)"$. #定义发送数据的时间间隔 interval=60 #定义计算与发送的模块名 sink_module=zabbix_sink sink_service=cacheL2 #定义需要的key sink_zabbix_monitor_keys=200,300,400,500 #定义发送给zabbix写数据的文件 sink_zabbix_send_file=/tmp/zabbix_send_info #定义发送zabbix sender路径 sink_zabbix_sender=/usr/bin/zabbix_sender #定义zabbix的配置文件 sink_zabbix_conf=/etc/zabbix/zabbix_agentd.conf. #定义输出log的格式,级别,路径等,方便调试程序。 logging_format=%(asctime)s %(filename)s [funcname:%(funcName)s] [line:%(lineno)d] %(levelname)s %(message)s logging_level=20 logging_filename=/tmp/plog.log . source_module=self-define-script-name 自定义source的具体实现,参看source module下的plog/source/youself_define_source.py. 在这个部分,主要是对数据流的处理,你同样需要写一个 Python的脚本,名字随意你定,但是你需要写到 plog.conf 中,默认有解析python正则的模块可以直接使用,类似下方: channel_module=filter_log 同样的你需要实现的channel可以参见 plog/channel/youself_define_channel.py. 这里实现了另外的一个grok channel模块,底层调用的是pygrok库,如果你对logstash比较熟悉,那么你应该可以比较灵活的使用Plog的grok channel。但在此之前你要确保你的系统已经安装了regex模块。在$Plog_HOME/conf/plog.conf中指定使用grok channel模块,并添加相应日志解析的grok pattern。比如像下面这样:. channel_module=grok_channel channel_filter_grok=%{HOSTNAME}\s%{DATA}\s%{NUMBER:response_time}\s%{WORD}\s\[.\]\s\".\"\s%{WORD:response_code} . 在这个部分,你同样需要写一个Python脚本,他的名字同样取决于你的个人喜好,你需要的是把你写的那个插件的名字写到plog.conf,例如下方: sink_module=cacheL2get_monitor 同样的你需要完成的脚本可以参见plog/sink/youself_define_sink.py. 下面的测试是读取一个本地本件,解析,计算自己要想的结果发送到zabbix监控系统。 1.git clone 4.cd .. && python plog.py -c conf/plog.conf. 5.you will see a file/tmp/zabbix_send_info_test123,its contents like followings: [xxxx@test Plog]$ cat /tmp/zabbix_send_info_test123 xxxx test123_300 0.000000 xxxx test123_200 59.000000 xxxx test123_500 0.000000 xxxx test123_400 0.000000 .
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              Plog has a low active ecosystem.
              It has 153 star(s) with 62 fork(s). There are 31 watchers for this library.
              OutlinedDot
              It had no major release in the last 6 months.
              There are 0 open issues and 1 have been closed. On average issues are closed in 507 days. There are no pull requests.
              It has a neutral sentiment in the developer community.
              The latest version of Plog is current.

            kandi-Quality Quality

              Plog has 0 bugs and 0 code smells.

            kandi-Security Security

              Plog has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              Plog code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              Plog does not have a standard license declared.
              Check the repository for any license declaration and review the terms closely.
              OutlinedDot
              Without a license, all rights are reserved, and you cannot use the library in your applications.

            kandi-Reuse Reuse

              Plog releases are not available. You will need to build from source code and install.
              Plog has no build file. You will be need to create the build yourself to build the component from source.
              Installation instructions are not available. Examples and code snippets are available.
              Plog saves you 174 person hours of effort in developing the same functionality from scratch.
              It has 432 lines of code, 41 functions and 20 files.
              It has low code complexity. Code complexity directly impacts maintainability of the code.

            Top functions reviewed by kandi - BETA

            kandi has reviewed Plog and discovered the below as its top functions. This is intended to give you an instant insight into Plog implemented functionality, and help decide if they suit your requirements.
            • Generate Grok Match object
            • Load patterns from file
            • Load all patterns from a directory
            • Run a plog worker
            • Read the configuration file
            • Deal with zabbix send
            • Write data to the logger
            Get all kandi verified functions for this library.

            Plog Key Features

            No Key Features are available at this moment for Plog.

            Plog Examples and Code Snippets

            No Code Snippets are available at this moment for Plog.

            Community Discussions

            QUESTION

            How to find missing ppp commands?
            Asked 2021-Mar-01 at 03:40

            I made my own linux distribution with buildroot. In "make menuconfig" and "make linux-menuconfig" I checked all options related to "ppp" and "pppd". Unfortunately, after building my distribution I can't use the commands "pon", "poff", "plog" and probably others. The system cannot see them. I looked and they are not in "/ usr / bin" or "/ usr / sbin". What could be causing this? I found out somewhere that these are debian-only commands, but how can I enable the pppd daemon?

            ...

            ANSWER

            Answered 2021-Mar-01 at 03:40

            ppd, pppd, pon, and etc are userland applications you need to install them using buildroot. So far you have enabled support in the kernel for ppp but you have not installed the actual application that manages the ppp connections which is pppd

            If build root doesn't already have a package for ppp/pppd, you can make a recipe to do that. The official website is: https://ppp.samba.org/

            pon, poff, and plog can be found in the scripts directory of the sources code.

            Source https://stackoverflow.com/questions/66377025

            QUESTION

            Dockerized React app not recompiling code
            Asked 2021-Jan-22 at 18:00

            I'm trying to dockerize a basic CRA template created through npx create-react-app project-name, of which Dockerfile would look like:

            ...

            ANSWER

            Answered 2021-Jan-22 at 16:42

            i think webpack server doesn't see any new changes, because you modify your local file, but container uses its copies in runtime, which was passed in build time. so you should mount your local dir to container.

            i can suggest you use docker-compose to mount your work dir from host to container:

            docker-compose.yml

            Source https://stackoverflow.com/questions/65848833

            QUESTION

            Why are class names in com.google.android.play.core obfuscated/minified when I'm not using proguard/R8?
            Asked 2021-Jan-07 at 07:12

            I have the following code in my app:

            ...

            ANSWER

            Answered 2021-Jan-07 at 07:12

            The reason for the class name c in the Google Play library is that the library itself has been minified before it is distributed through Maven.

            The aar can be found here, and the classes.jar inside of it already has the minified names.

            Source https://stackoverflow.com/questions/65588560

            QUESTION

            c++ thread worker failure under high load
            Asked 2020-Aug-31 at 09:51

            I have been working on a idea for a system where I can have many workers that are triggered on a regular basis by a a central timer class. The part I'm concerned about here is a TriggeredWorker which, in a loop, uses the mutex & conditionVariable approach to wait to be told to do work. It has a method trigger that is called (by a different thread) that triggers work to be done. It is an abstract class that has to be subclassed for the actual work method to be implemented.

            I have a test that shows that this mechanism works. However, as I increase the load by reducing the trigger interval, the test starts to fail. When I delay 20 microseconds between triggers, the test is 100% reliable. As I reduce down to 1 microsecond, I start to get failures in that the count of work performed reduces from 1000 (expected) to values like 986, 933, 999 etc..

            My questions are: (1) what is it that is going wrong and how can I capture what is going wrong so I can report it or do something about it? And, (2) is there some better approach that I could use that would be better? I have to admit that my experience with c++ is limited to the last 3 months, although I have worked with other languages for several years.

            Many thanks for reading...

            Here are the key bits of code:

            Triggered worker header file:

            ...

            ANSWER

            Answered 2020-Aug-31 at 09:51

            What happens when worker.trigger() is called twice before workLoop acquires the lock? You loose one of those "triggers". Smaller time gap means higher probability of test failure, because of higher probability of multiple consecutive worker.trigger() calls before workLoop wakes up. Note that there's nothing that guarantees that workLoop will acquire the lock after worker.trigger() but before another worker.trigger() happens, even when those calls happen one after another (i.e. not in parallel). This is governed by the OS scheduler and we have no control over it.

            Anyway the core problem is that setting ready_ = true twice looses information. Unlike incrementing an integer twice. And so the simplest solution is to replace bool with int and do inc/dec with == 0 checks. This solution is also known as semaphore. More advanced (potentially better, especially when you need to pass some data to the worker) approach is to use a (bounded?) thread safe queue. That depends on what exactly you are trying to achieve.

            BTW 1: all your reads and updates, except for stop() function (and start() but this isn't really relevant), happen under the lock. I suggest you fix stop() to be under lock as well (since it is rarely called anyway) and turn atomics into non-atomics. There's an unnecessary overhead of atomics at the moment.

            BTW 2: I suggest not using thread.detach(). You should store the std::thread object on TriggeredWorker and add destructor that does stop with join. These are not independent beings and so without detach() you make your code safer (one should never die without the other).

            Source https://stackoverflow.com/questions/63668081

            QUESTION

            'data.forEach is not a function' when loading csv in d3.js
            Asked 2020-Aug-27 at 22:08

            I am building a simple feedback system for a study. I have just started to learn JS and D3.js, but decided to build some very simple plot in D3.js. However, I get the following error message and I don't understand how I can fix it?

            (index):36 Uncaught (in promise) TypeError: data.forEach is not a function

            What I want to do with this script is to take a single line in a .csv and put the number on the screen. This my code

            ...

            ANSWER

            Answered 2020-Aug-27 at 21:49

            According to the docs, the function you're passing to d3.csv() is not a callback, but a row conversion function - what you're passed is not (error, data) but just d (the row being converted). You should skip the data.forEach and use the row conversion function for that, and put the rest of the code in the .then() callback.

            Source https://stackoverflow.com/questions/63624508

            QUESTION

            How to create a genome-wide reads density map in R (for a bacterial genome)
            Asked 2020-Apr-24 at 22:58

            I have a data frame (pLog) containing the number of reads per nucleotide for a chip-seq experiment done for a E. coli genome (4.6MB). I want to be able to plot on the X axis the chromosomal position and on the Y axis the number of reads. To make it easier, I binned the data in windows of 100bp. That makes the data frame of 46,259 rows and 2 columns. One column is named "position" and has a number representing a chromosomal position (1,101,201,....) and the other column is named "values" and contains the number of reads found on that bin e.g.(210,511,315,....). I have been using ggplot for all my analysis and I would like to use it for this plot, if possible.

            I am trying for the graph to look something like this:

            but I haven't been able to plot it.

            This is how my data looks like

            I tried

            ...

            ANSWER

            Answered 2020-Apr-24 at 22:58

            You cannot use geom_histogram(), try geom_line:

            Source https://stackoverflow.com/questions/61416410

            QUESTION

            How do I use a non-exponential backoff strategy for retries in python requests?
            Asked 2020-Mar-09 at 12:59

            I want to make use of the requests module backoff strategy outlined at https://stackoverflow.com/a/35504626/1021819, the code reproduced from that answer being:

            ...

            ANSWER

            Answered 2020-Mar-09 at 12:04

            Although I was deep in enemy territory, I ended up superclassing Retry and overloading Retry.get_backoff_time(). The new class MyRetry now takes an optional lambda specifying how to calculate the backoff time.

            Source https://stackoverflow.com/questions/60577683

            QUESTION

            I have some troubles with memory
            Asked 2019-Dec-29 at 04:29

            Hello mates and sorry for my english! Firstly, could I ask help at this forum with my bugs in case I cant figure out the problem on my own?

            So to begin with, i have a file that contains some logs that are structured like that:

            ...

            ANSWER

            Answered 2019-Dec-29 at 04:29

            I have changed the code, which should now work! Please see comments to see what has been changed.

            Source https://stackoverflow.com/questions/59512616

            QUESTION

            How to launch Informix via testcontainers?
            Asked 2019-Oct-11 at 17:31

            I'm very interested in using testcontainers in my project.

            However, I'm having a hard time setting it up to work with Informix.

            Note that I can start an informix container using Docker-for-Mac and it will build and start.

            Not sure it can work with testcontainers though. I wish it would.

            Here's what I have so far

            Test class

            ...

            ANSWER

            Answered 2019-Oct-11 at 17:31

            There was a mis-configuration in the docker image for Informix. The servers that start in the docker containers only would listen on the hostname, and not localhost. Testcontainers uses 'localhost' as the network interface to connect to your containers. So when you used .withExposedPorts(9088) the port was not actually exposed on a network interface TestContainers could connect to.

            This is why you still had trouble most likely even if you waited for the log message you also waited on the port and it would never be available.

            The good news is that this is now fixed and available by pulling down the latest Informix docker images

            ibmcom/informix-developer-database:latest for the latest 14.10 docker image

            Below is the code I ran to validate the new images are working better with TestContainers.

            Source https://stackoverflow.com/questions/56910441

            QUESTION

            For each loop is not starting with the latest E-mail in the inbox, its starting with an older E-mail
            Asked 2019-Oct-08 at 06:45

            I am writing an excel vba code which will loop through specified folder in outlook and get the "MailItem"s for further processing. It was working fine until there was an upgrade of MS Office suite to 2016. Now, when the loop starts, it picks the mail that was there just before the office upgrade. Why its not beginning with the latest mail that is in the folder?

            Adding the code here.

            I have tried to create a new macro enabled excel file and typed the code and ran it. Result is same.

            ...

            ANSWER

            Answered 2019-Oct-07 at 19:14

            You were fortunate the mail previously processed latest first. To be sure about the order you must sort.

            Source https://stackoverflow.com/questions/58274460

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install Plog

            You can download it from GitHub.
            You can use Plog like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

            Support

            For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
            CLONE
          • HTTPS

            https://github.com/CNSRE/Plog.git

          • CLI

            gh repo clone CNSRE/Plog

          • sshUrl

            git@github.com:CNSRE/Plog.git

          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link