grafana | composable observability and data visualization platform | Dashboard library

 by   grafana TypeScript Version: v8.5.22 License: AGPL-3.0

kandi X-RAY | grafana Summary

grafana is a TypeScript library typically used in Analytics, Dashboard, Prometheus, Grafana applications. grafana has no bugs, it has a Strong Copyleft License and it has medium support. However grafana has 6 vulnerabilities. You can download it from GitHub.
The open-source platform for monitoring and observability.
    Support
      Quality
        Security
          License
            Reuse
            Support
              Quality
                Security
                  License
                    Reuse

                      kandi-support Support

                        summary
                        grafana has a medium active ecosystem.
                        summary
                        It has 54610 star(s) with 10643 fork(s). There are 1275 watchers for this library.
                        summary
                        There were 10 major release(s) in the last 6 months.
                        summary
                        There are 3000 open issues and 25919 have been closed. On average issues are closed in 115 days. There are 320 open pull requests and 0 closed requests.
                        summary
                        It has a neutral sentiment in the developer community.
                        summary
                        The latest version of grafana is v8.5.22
                        grafana Support
                          Best in #Dashboard
                            Average in #Dashboard
                            grafana Support
                              Best in #Dashboard
                                Average in #Dashboard

                                  kandi-Quality Quality

                                    summary
                                    grafana has 0 bugs and 0 code smells.
                                    grafana Quality
                                      Best in #Dashboard
                                        Average in #Dashboard
                                        grafana Quality
                                          Best in #Dashboard
                                            Average in #Dashboard

                                              kandi-Security Security

                                                summary
                                                grafana has 6 vulnerability issues reported (0 critical, 1 high, 5 medium, 0 low).
                                                summary
                                                grafana code analysis shows 0 unresolved vulnerabilities.
                                                summary
                                                There are 0 security hotspots that need review.
                                                grafana Security
                                                  Best in #Dashboard
                                                    Average in #Dashboard
                                                    grafana Security
                                                      Best in #Dashboard
                                                        Average in #Dashboard

                                                          kandi-License License

                                                            summary
                                                            grafana is licensed under the AGPL-3.0 License. This license is Strong Copyleft.
                                                            summary
                                                            Strong Copyleft licenses enforce sharing, and you can use them when creating open source projects.
                                                            grafana License
                                                              Best in #Dashboard
                                                                Average in #Dashboard
                                                                grafana License
                                                                  Best in #Dashboard
                                                                    Average in #Dashboard

                                                                      kandi-Reuse Reuse

                                                                        summary
                                                                        grafana releases are available to install and integrate.
                                                                        summary
                                                                        Installation instructions are available. Examples and code snippets are not available.
                                                                        summary
                                                                        It has 236850 lines of code, 7462 functions and 5741 files.
                                                                        summary
                                                                        It has medium code complexity. Code complexity directly impacts maintainability of the code.
                                                                        grafana Reuse
                                                                          Best in #Dashboard
                                                                            Average in #Dashboard
                                                                            grafana Reuse
                                                                              Best in #Dashboard
                                                                                Average in #Dashboard
                                                                                  Top functions reviewed by kandi - BETA
                                                                                  kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
                                                                                  Currently covering the most popular Java, JavaScript and Python libraries. See a Sample Here
                                                                                  Get all kandi verified functions for this library.
                                                                                  Get all kandi verified functions for this library.

                                                                                  grafana Key Features

                                                                                  Visualize: Fast and flexible client side graphs with a multitude of options. Panel plugins offer many different ways to visualize metrics and logs.
                                                                                  Dynamic Dashboards: Create dynamic & reusable dashboards with template variables that appear as dropdowns at the top of the dashboard.
                                                                                  Explore Metrics: Explore your data through ad-hoc queries and dynamic drilldown. Split view and compare different time ranges, queries and data sources side by side.
                                                                                  Explore Logs: Experience the magic of switching from metrics to logs with preserved label filters. Quickly search through all your logs or streaming them live.
                                                                                  Alerting: Visually define alert rules for your most important metrics. Grafana will continuously evaluate and send notifications to systems like Slack, PagerDuty, VictorOps, OpsGenie.
                                                                                  Mixed Data Sources: Mix different data sources in the same graph! You can specify a data source on a per-query basis. This works for even custom datasources.

                                                                                  grafana Examples and Code Snippets

                                                                                  How to make Grafana's datasource plugin compliant with Query Inspector?
                                                                                  JavaScriptdot imgLines of Code : 14dot imgLicense : Strong Copyleft (CC BY-SA 4.0)
                                                                                  copy iconCopy
                                                                                  import { getBackendSrv } from '@grafana/runtime';
                                                                                  
                                                                                  //later in your code
                                                                                  getBackendSrv().datasourceRequest({
                                                                                        url:'https://api.github.com/repos/grafana/grafana/stats/commit_activity',
                                                                                        method:'GET'
                                                                                      }).then((data: any) => console.log('DATA',data));
                                                                                  
                                                                                  import { SystemJS } from '@grafana/runtime'
                                                                                  //later in your code
                                                                                  SystemJS.load('app/core/app_events').then((appEvents:any) => {
                                                                                      appEvents.emit('ds-request-response', data) // where data is the data to display
                                                                                  })
                                                                                  
                                                                                  Community Discussions

                                                                                  Trending Discussions on grafana

                                                                                  Remove a part of a log in Loki
                                                                                  chevron right
                                                                                  How can you integrate grafana with Google Cloud SQL
                                                                                  chevron right
                                                                                  Enable use of images from the local library on Kubernetes
                                                                                  chevron right
                                                                                  Understanding the CPU Busy Prometheus query
                                                                                  chevron right
                                                                                  Thanos-Query/Query-Frontend does not show any metrics
                                                                                  chevron right
                                                                                  Add Kubernetes scrape target to Prometheus instance that is NOT in Kubernetes
                                                                                  chevron right
                                                                                  Sucessfully queries the azure monitor service. Workspace not found. While using azuremarket place Grafana
                                                                                  chevron right
                                                                                  Grafana - Is it possible to use variables in Loki-based dashboard query?
                                                                                  chevron right
                                                                                  PostgreSQL Default Result Limit
                                                                                  chevron right
                                                                                  Trigger Beam ParDo at window closing only
                                                                                  chevron right

                                                                                  QUESTION

                                                                                  Remove a part of a log in Loki
                                                                                  Asked 2022-Mar-21 at 10:18

                                                                                  I have installed Grafana, Loki, Promtail and Prometheus with the grafana/loki-stack.

                                                                                  I also have Nginx set up with the Nginx helm chart.

                                                                                  Promtail is ingesting logs fine into Loki, but I want to customise the way my logs look. Specifically I want to remove a part of the log because it creates errors when trying to parse it with either logfmt or json (Error: LogfmtParserErr and Error: JsonParserErr respectively).

                                                                                  The logs look like this:

                                                                                  2022-02-21T13:41:53.155640208Z stdout F timestamp=2022-02-21T13:41:53+00:00 http_request_method=POST http_response_status_code=200 http_response_time=0.001 http_version=HTTP/2.0 http_request_body_bytes=0 http_request_bytes=63
                                                                                  

                                                                                  and I want to remove the part where it says stdout F so the log will look like this:

                                                                                  2022-02-21T13:41:53.155640208Z timestamp=2022-02-21T13:41:53+00:00 http_request_method=POST http_response_status_code=200 http_response_time=0.001 http_version=HTTP/2.0 http_request_body_bytes=0 http_request_bytes=63
                                                                                  

                                                                                  I have figured out that on the ingestion side it could be something with Promtail, but ist it also possible to make a LogQL query in Loki to just replace that string? And how would one set up the Promtail configuration for the wanted behaviour?

                                                                                  ANSWER

                                                                                  Answered 2022-Feb-21 at 17:57

                                                                                  Promtail should be configured to replace the string with the replace stage.

                                                                                  Here is a sample config that removes the stdout F part of the log for all logs coming from the namespace ingress.

                                                                                  promtail:
                                                                                    enabled: true
                                                                                    pipelineStages:
                                                                                    - docker: {}
                                                                                    - match:
                                                                                        selector: '{namespace="ingress"}'
                                                                                        stages:
                                                                                        - replace:
                                                                                            expression: "(stdout F)"
                                                                                            replace: ""
                                                                                  

                                                                                  Specifically this example works for the grafana/loki-stack chart.

                                                                                  Source https://stackoverflow.com/questions/71210935

                                                                                  QUESTION

                                                                                  How can you integrate grafana with Google Cloud SQL
                                                                                  Asked 2022-Mar-21 at 05:50

                                                                                  I haven't been able to find how to take a Postgres instance on Google Cloud SQL (on GCP) and hook it up to a grafana dashboard to visualize the data that is in the DB. Is there an accepted easy way to do this? I'm a complete newbie to grafana and have limited experience with GCP(used cloud sql proxy to connect to a postgres instance)

                                                                                  ANSWER

                                                                                  Answered 2022-Mar-20 at 18:50

                                                                                  Grafana display the data. Google Cloud Monitoring store the data to display. So, you have to make a link between both.

                                                                                  And boom, magically, a plug-in exists!

                                                                                  Note: when you know what you search, it's easier to find it. Understand your architecture to reach the next level!

                                                                                  Source https://stackoverflow.com/questions/71547327

                                                                                  QUESTION

                                                                                  Enable use of images from the local library on Kubernetes
                                                                                  Asked 2022-Mar-20 at 13:23

                                                                                  I'm following a tutorial https://docs.openfaas.com/tutorials/first-python-function/,

                                                                                  currently, I have the right image

                                                                                  $ docker images | grep hello-openfaas
                                                                                  wm/hello-openfaas                                     latest                          bd08d01ce09b   34 minutes ago      65.2MB
                                                                                  $ faas-cli deploy -f ./hello-openfaas.yml 
                                                                                  Deploying: hello-openfaas.
                                                                                  WARNING! You are not using an encrypted connection to the gateway, consider using HTTPS.
                                                                                  
                                                                                  Deployed. 202 Accepted.
                                                                                  URL: http://IP:8099/function/hello-openfaas
                                                                                  

                                                                                  there is a step that forewarns me to do some setup(My case is I'm using Kubernetes and minikube and don't want to push to a remote container registry, I should enable the use of images from the local library on Kubernetes.), I see the hints

                                                                                  see the helm chart for how to set the ImagePullPolicy
                                                                                  

                                                                                  I'm not sure how to configure it correctly. the final result indicates I failed.

                                                                                  Unsurprisingly, I couldn't access the function service, I find some clues in https://docs.openfaas.com/deployment/troubleshooting/#openfaas-didnt-start which might help to diagnose the problem.

                                                                                  $ kubectl logs -n openfaas-fn deploy/hello-openfaas
                                                                                  Error from server (BadRequest): container "hello-openfaas" in pod "hello-openfaas-558f99477f-wd697" is waiting to start: trying and failing to pull image
                                                                                  
                                                                                  $ kubectl describe -n openfaas-fn deploy/hello-openfaas
                                                                                  Name:                   hello-openfaas
                                                                                  Namespace:              openfaas-fn
                                                                                  CreationTimestamp:      Wed, 16 Mar 2022 14:59:49 +0800
                                                                                  Labels:                 faas_function=hello-openfaas
                                                                                  Annotations:            deployment.kubernetes.io/revision: 1
                                                                                                          prometheus.io.scrape: false
                                                                                  Selector:               faas_function=hello-openfaas
                                                                                  Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
                                                                                  StrategyType:           RollingUpdate
                                                                                  MinReadySeconds:        0
                                                                                  RollingUpdateStrategy:  0 max unavailable, 1 max surge
                                                                                  Pod Template:
                                                                                    Labels:       faas_function=hello-openfaas
                                                                                    Annotations:  prometheus.io.scrape: false
                                                                                    Containers:
                                                                                     hello-openfaas:
                                                                                      Image:      wm/hello-openfaas:latest
                                                                                      Port:       8080/TCP
                                                                                      Host Port:  0/TCP
                                                                                      Liveness:   http-get http://:8080/_/health delay=2s timeout=1s period=2s #success=1 #failure=3
                                                                                      Readiness:  http-get http://:8080/_/health delay=2s timeout=1s period=2s #success=1 #failure=3
                                                                                      Environment:
                                                                                        fprocess:  python3 index.py
                                                                                      Mounts:      
                                                                                    Volumes:       
                                                                                  Conditions:
                                                                                    Type           Status  Reason
                                                                                    ----           ------  ------
                                                                                    Available      False   MinimumReplicasUnavailable
                                                                                    Progressing    False   ProgressDeadlineExceeded
                                                                                  OldReplicaSets:  
                                                                                  NewReplicaSet:   hello-openfaas-558f99477f (1/1 replicas created)
                                                                                  Events:
                                                                                    Type    Reason             Age   From                   Message
                                                                                    ----    ------             ----  ----                   -------
                                                                                    Normal  ScalingReplicaSet  29m   deployment-controller  Scaled up replica set hello-openfaas-558f99477f to 1
                                                                                  

                                                                                  hello-openfaas.yml

                                                                                  version: 1.0
                                                                                  provider:
                                                                                    name: openfaas
                                                                                    gateway: http://IP:8099
                                                                                  functions:
                                                                                    hello-openfaas:
                                                                                      lang: python3
                                                                                      handler: ./hello-openfaas
                                                                                      image: wm/hello-openfaas:latest
                                                                                      imagePullPolicy: Never
                                                                                  

                                                                                  I create a new project hello-openfaas2 to reproduce this error

                                                                                  $ faas-cli new --lang python3 hello-openfaas2 --prefix="wm"
                                                                                  Folder: hello-openfaas2 created.
                                                                                  # I add `imagePullPolicy: Never` to `hello-openfaas2.yml`
                                                                                  $ faas-cli build -f ./hello-openfaas2.yml 
                                                                                  $ faas-cli deploy -f ./hello-openfaas2.yml 
                                                                                  Deploying: hello-openfaas2.
                                                                                  WARNING! You are not using an encrypted connection to the gateway, consider using HTTPS.
                                                                                  
                                                                                  Deployed. 202 Accepted.
                                                                                  URL: http://192.168.1.3:8099/function/hello-openfaas2
                                                                                  
                                                                                  
                                                                                  $ kubectl logs -n openfaas-fn deploy/hello-openfaas2
                                                                                  Error from server (BadRequest): container "hello-openfaas2" in pod "hello-openfaas2-7c67488865-7d7vm" is waiting to start: image can't be pulled
                                                                                  
                                                                                  $ kubectl get pods --all-namespaces
                                                                                  NAMESPACE              NAME                                        READY   STATUS             RESTARTS         AGE
                                                                                  kube-system            coredns-64897985d-kp7vf                     1/1     Running            0                47h
                                                                                  ...
                                                                                  openfaas-fn            env-6c79f7b946-bzbtm                        1/1     Running            0                4h28m
                                                                                  openfaas-fn            figlet-54db496f88-957xl                     1/1     Running            0                18h
                                                                                  openfaas-fn            hello-openfaas-547857b9d6-z277c             0/1     ImagePullBackOff   0                127m
                                                                                  openfaas-fn            hello-openfaas-7b6946b4f9-hcvq4             0/1     ImagePullBackOff   0                165m
                                                                                  openfaas-fn            hello-openfaas2-7c67488865-qmrkl            0/1     ImagePullBackOff   0                13m
                                                                                  openfaas-fn            hello-openfaas3-65847b8b67-b94kd            0/1     ImagePullBackOff   0                97m
                                                                                  openfaas-fn            hello-python-554b464498-zxcdv               0/1     ErrImagePull       0                3h23m
                                                                                  openfaas-fn            hello-python-8698bc68bd-62gh9               0/1     ImagePullBackOff   0                3h25m
                                                                                  

                                                                                  from https://docs.openfaas.com/reference/yaml/, I know I put the imagePullPolicy in the wrong place, there is no such keyword in its schema.

                                                                                  I also tried eval $(minikube docker-env and still get the same error.

                                                                                  I've a feeling that faas-cli deploy can be replace by helm, they all mean to run the image(whether from remote or local) in Kubernetes cluster, then I can use helm chart to setup the pullPolicy there. Even though the detail is not still clear to me, This discovery inspires me.

                                                                                  So far, after eval $(minikube docker-env)

                                                                                  $ docker images
                                                                                  REPOSITORY                                TAG        IMAGE ID       CREATED             SIZE
                                                                                  wm/hello-openfaas2                        0.1        03c21bd96d5e   About an hour ago   65.2MB
                                                                                  python                                    3-alpine   69fba17b9bae   12 days ago         48.6MB
                                                                                  ghcr.io/openfaas/figlet                   latest     ca5eef0de441   2 weeks ago         14.8MB
                                                                                  ghcr.io/openfaas/alpine                   latest     35f3d4be6bb8   2 weeks ago         14.2MB
                                                                                  ghcr.io/openfaas/faas-netes               0.14.2     524b510505ec   3 weeks ago         77.3MB
                                                                                  k8s.gcr.io/kube-apiserver                 v1.23.3    f40be0088a83   7 weeks ago         135MB
                                                                                  k8s.gcr.io/kube-controller-manager        v1.23.3    b07520cd7ab7   7 weeks ago         125MB
                                                                                  k8s.gcr.io/kube-scheduler                 v1.23.3    99a3486be4f2   7 weeks ago         53.5MB
                                                                                  k8s.gcr.io/kube-proxy                     v1.23.3    9b7cc9982109   7 weeks ago         112MB
                                                                                  ghcr.io/openfaas/gateway                  0.21.3     ab4851262cd1   7 weeks ago         30.6MB
                                                                                  ghcr.io/openfaas/basic-auth               0.21.3     16e7168a17a3   7 weeks ago         14.3MB
                                                                                  k8s.gcr.io/etcd                           3.5.1-0    25f8c7f3da61   4 months ago        293MB
                                                                                  ghcr.io/openfaas/classic-watchdog         0.2.0      6f97aa96da81   4 months ago        8.18MB
                                                                                  k8s.gcr.io/coredns/coredns                v1.8.6     a4ca41631cc7   5 months ago        46.8MB
                                                                                  k8s.gcr.io/pause                          3.6        6270bb605e12   6 months ago        683kB
                                                                                  ghcr.io/openfaas/queue-worker             0.12.2     56e7216201bc   7 months ago        7.97MB
                                                                                  kubernetesui/dashboard                    v2.3.1     e1482a24335a   9 months ago        220MB
                                                                                  kubernetesui/metrics-scraper              v1.0.7     7801cfc6d5c0   9 months ago        34.4MB
                                                                                  nats-streaming                            0.22.0     12f2d32e0c9a   9 months ago        19.8MB
                                                                                  gcr.io/k8s-minikube/storage-provisioner   v5         6e38f40d628d   11 months ago       31.5MB
                                                                                  functions/markdown-render                 latest     93b5da182216   2 years ago         24.6MB
                                                                                  functions/hubstats                        latest     01affa91e9e4   2 years ago         29.3MB
                                                                                  functions/nodeinfo                        latest     2fe8a87bf79c   2 years ago         71.4MB
                                                                                  functions/alpine                          latest     46c6f6d74471   2 years ago         21.5MB
                                                                                  prom/prometheus                           v2.11.0    b97ed892eb23   2 years ago         126MB
                                                                                  prom/alertmanager                         v0.18.0    ce3c87f17369   2 years ago         51.9MB
                                                                                  alexellis2/openfaas-colorization          0.4.1      d36b67b1b5c1   2 years ago         1.84GB
                                                                                  rorpage/text-to-speech                    latest     5dc20810eb54   2 years ago         86.9MB
                                                                                  stefanprodan/faas-grafana                 4.6.3      2a4bd9caea50   4 years ago         284MB
                                                                                  
                                                                                  $ kubectl get pods --all-namespaces
                                                                                  NAMESPACE              NAME                                        READY   STATUS             RESTARTS        AGE
                                                                                  kube-system            coredns-64897985d-kp7vf                     1/1     Running            0               6d
                                                                                  kube-system            etcd-minikube                               1/1     Running            0               6d
                                                                                  kube-system            kube-apiserver-minikube                     1/1     Running            0               6d
                                                                                  kube-system            kube-controller-manager-minikube            1/1     Running            0               6d
                                                                                  kube-system            kube-proxy-5m8lr                            1/1     Running            0               6d
                                                                                  kube-system            kube-scheduler-minikube                     1/1     Running            0               6d
                                                                                  kube-system            storage-provisioner                         1/1     Running            1 (6d ago)      6d
                                                                                  kubernetes-dashboard   dashboard-metrics-scraper-58549894f-97tsv   1/1     Running            0               5d7h
                                                                                  kubernetes-dashboard   kubernetes-dashboard-ccd587f44-lkwcx        1/1     Running            0               5d7h
                                                                                  openfaas-fn            base64-6bdbcdb64c-djz8f                     1/1     Running            0               5d1h
                                                                                  openfaas-fn            colorise-85c74c686b-2fz66                   1/1     Running            0               4d5h
                                                                                  openfaas-fn            echoit-5d7df6684c-k6ljn                     1/1     Running            0               5d1h
                                                                                  openfaas-fn            env-6c79f7b946-bzbtm                        1/1     Running            0               4d5h
                                                                                  openfaas-fn            figlet-54db496f88-957xl                     1/1     Running            0               4d19h
                                                                                  openfaas-fn            hello-openfaas-547857b9d6-z277c             0/1     ImagePullBackOff   0               4d3h
                                                                                  openfaas-fn            hello-openfaas-7b6946b4f9-hcvq4             0/1     ImagePullBackOff   0               4d3h
                                                                                  openfaas-fn            hello-openfaas2-5c6f6cb5d9-24hkz            0/1     ImagePullBackOff   0               9m22s
                                                                                  openfaas-fn            hello-openfaas2-8957bb47b-7cgjg             0/1     ImagePullBackOff   0               2d22h
                                                                                  openfaas-fn            hello-openfaas3-65847b8b67-b94kd            0/1     ImagePullBackOff   0               4d2h
                                                                                  openfaas-fn            hello-python-6d6976845f-cwsln               0/1     ImagePullBackOff   0               3d19h
                                                                                  openfaas-fn            hello-python-b577cb8dc-64wf5                0/1     ImagePullBackOff   0               3d9h
                                                                                  openfaas-fn            hubstats-b6cd4dccc-z8tvl                    1/1     Running            0               5d1h
                                                                                  openfaas-fn            markdown-68f69f47c8-w5m47                   1/1     Running            0               5d1h
                                                                                  openfaas-fn            nodeinfo-d48cbbfcc-hfj79                    1/1     Running            0               5d1h
                                                                                  openfaas-fn            openfaas2-fun                               1/1     Running            0               15s
                                                                                  openfaas-fn            text-to-speech-74ffcdfd7-997t4              0/1     CrashLoopBackOff   2235 (3s ago)   4d5h
                                                                                  openfaas-fn            wordcount-6489865566-cvfzr                  1/1     Running            0               5d1h
                                                                                  openfaas               alertmanager-88449c789-fq2rg                1/1     Running            0               3d1h
                                                                                  openfaas               basic-auth-plugin-75fd7d69c5-zw4jh          1/1     Running            0               3d2h
                                                                                  openfaas               gateway-5c4bb7c5d7-n8h27                    2/2     Running            0               3d2h
                                                                                  openfaas               grafana                                     1/1     Running            0               4d8h
                                                                                  openfaas               nats-647b476664-hkr7p                       1/1     Running            0               3d2h
                                                                                  openfaas               prometheus-687648749f-tl8jp                 1/1     Running            0               3d1h
                                                                                  openfaas               queue-worker-7777ffd7f6-htx6t               1/1     Running            0               3d2h
                                                                                  
                                                                                  
                                                                                  $ kubectl get -o yaml -n openfaas-fn deploy/hello-openfaas2
                                                                                  apiVersion: apps/v1
                                                                                  kind: Deployment
                                                                                  metadata:
                                                                                    annotations:
                                                                                      deployment.kubernetes.io/revision: "6"
                                                                                      prometheus.io.scrape: "false"
                                                                                    creationTimestamp: "2022-03-17T12:47:35Z"
                                                                                    generation: 6
                                                                                    labels:
                                                                                      faas_function: hello-openfaas2
                                                                                    name: hello-openfaas2
                                                                                    namespace: openfaas-fn
                                                                                    resourceVersion: "400833"
                                                                                    uid: 9c4e9d26-23af-4f93-8538-4e2d96f0d7e0
                                                                                  spec:
                                                                                    progressDeadlineSeconds: 600
                                                                                    replicas: 1
                                                                                    revisionHistoryLimit: 10
                                                                                    selector:
                                                                                      matchLabels:
                                                                                        faas_function: hello-openfaas2
                                                                                    strategy:
                                                                                      rollingUpdate:
                                                                                        maxSurge: 1
                                                                                        maxUnavailable: 0
                                                                                      type: RollingUpdate
                                                                                    template:
                                                                                      metadata:
                                                                                        annotations:
                                                                                          prometheus.io.scrape: "false"
                                                                                        creationTimestamp: null
                                                                                        labels:
                                                                                          faas_function: hello-openfaas2
                                                                                          uid: "969512830"
                                                                                        name: hello-openfaas2
                                                                                      spec:
                                                                                        containers:
                                                                                        - env:
                                                                                          - name: fprocess
                                                                                            value: python3 index.py
                                                                                          image: wm/hello-openfaas2:0.1
                                                                                          imagePullPolicy: Always
                                                                                          livenessProbe:
                                                                                            failureThreshold: 3
                                                                                            httpGet:
                                                                                              path: /_/health
                                                                                              port: 8080
                                                                                              scheme: HTTP
                                                                                            initialDelaySeconds: 2
                                                                                            periodSeconds: 2
                                                                                            successThreshold: 1
                                                                                            timeoutSeconds: 1
                                                                                          name: hello-openfaas2
                                                                                          ports:
                                                                                          - containerPort: 8080
                                                                                            name: http
                                                                                            protocol: TCP
                                                                                          readinessProbe:
                                                                                            failureThreshold: 3
                                                                                            httpGet:
                                                                                              path: /_/health
                                                                                              port: 8080
                                                                                              scheme: HTTP
                                                                                            initialDelaySeconds: 2
                                                                                            periodSeconds: 2
                                                                                            successThreshold: 1
                                                                                            timeoutSeconds: 1
                                                                                          resources: {}
                                                                                          securityContext:
                                                                                            allowPrivilegeEscalation: false
                                                                                            readOnlyRootFilesystem: false
                                                                                          terminationMessagePath: /dev/termination-log
                                                                                          terminationMessagePolicy: File
                                                                                        dnsPolicy: ClusterFirst
                                                                                        enableServiceLinks: false
                                                                                        restartPolicy: Always
                                                                                        schedulerName: default-scheduler
                                                                                        securityContext: {}
                                                                                        terminationGracePeriodSeconds: 30
                                                                                  status:
                                                                                    conditions:
                                                                                    - lastTransitionTime: "2022-03-17T12:47:35Z"
                                                                                      lastUpdateTime: "2022-03-17T12:47:35Z"
                                                                                      message: Deployment does not have minimum availability.
                                                                                      reason: MinimumReplicasUnavailable
                                                                                      status: "False"
                                                                                      type: Available
                                                                                    - lastTransitionTime: "2022-03-20T12:16:56Z"
                                                                                      lastUpdateTime: "2022-03-20T12:16:56Z"
                                                                                      message: ReplicaSet "hello-openfaas2-5d6c7c7fb4" has timed out progressing.
                                                                                      reason: ProgressDeadlineExceeded
                                                                                      status: "False"
                                                                                      type: Progressing
                                                                                    observedGeneration: 6
                                                                                    replicas: 2
                                                                                    unavailableReplicas: 2
                                                                                    updatedReplicas: 1
                                                                                  

                                                                                  In one shell,

                                                                                  docker@minikube:~$ docker run  --name wm -ti wm/hello-openfaas2:0.1
                                                                                  2022/03/20 13:04:52 Version: 0.2.0  SHA: 56bf6aac54deb3863a690f5fc03a2a38e7d9e6ef
                                                                                  2022/03/20 13:04:52 Timeouts: read: 5s write: 5s hard: 0s health: 5s.
                                                                                  2022/03/20 13:04:52 Listening on port: 8080
                                                                                  ...
                                                                                  
                                                                                  

                                                                                  and another shell

                                                                                  docker@minikube:~$ docker ps | grep wm
                                                                                  d7796286641c   wm/hello-openfaas2:0.1             "fwatchdog"              3 minutes ago       Up 3 minutes (healthy)   8080/tcp   wm
                                                                                  

                                                                                  ANSWER

                                                                                  Answered 2022-Mar-16 at 08:10

                                                                                  If your image has a latest tag, the Pod's ImagePullPolicy will be automatically set to Always. Each time the pod is created, Kubernetes tries to pull the newest image.

                                                                                  Try not tagging the image as latest or manually setting the Pod's ImagePullPolicy to Never. If you're using static manifest to create a Pod, the setting will be like the following:

                                                                                  containers:
                                                                                    - name: test-container
                                                                                      image: testImage:latest
                                                                                      imagePullPolicy: Never
                                                                                  

                                                                                  Source https://stackoverflow.com/questions/71493306

                                                                                  QUESTION

                                                                                  Understanding the CPU Busy Prometheus query
                                                                                  Asked 2022-Mar-19 at 12:37

                                                                                  I am new to Grafana and Prometheus. I have read a lot of documentation and now I"m trying to work backwards by reviewing some existing queries and making sure I understand them

                                                                                  I have downloaded the Node Exporter Full dashboard (https://grafana.com/grafana/dashboards/1860). I have been reviewing the CPU Busy query and I"m a bit confused. I am quoting it below, spaced out so we can see the nested sections better:

                                                                                  In this query, job is node-exporter while instance is the IP and port of the server. This is my base understanding of the query: node_cpu_seconds_total is a counter of the number of seconds the CPU took at a given sample.

                                                                                  1. Line 5: Get cpu seconds at a given instant, broken down by the individual CPU cores
                                                                                  2. Line 4: Add up all CPU seconds across all cores
                                                                                  3. Line 3: Why is there an additional count()? Does it do anything?
                                                                                  4. Line 12: Rate vector - get cpu seconds of when the cpu was idle over the given rate period
                                                                                  5. Line 11: Take a rate to transfer that into the rate of change of cpu seconds (and return an instant vector)
                                                                                  6. Line 10: Sum up all rates, broken down by CPU modes
                                                                                  7. Line 9: Take the single average rate across all CPU mode rates
                                                                                  8. Line 8: Subtract the average rate of change (Line 9) from total CPU seconds (Line 3)
                                                                                  9. Line 16: Multiple by 100 to convert minutes to seconds 10: Line 18-20: Divide Line 19 by the count of the count of all CPU seconds across all CPUs

                                                                                  My questions are as follows:

                                                                                  • I would have thought that CPU usage would simply be (all non idle cpu usage) / (total cpu usage). I therefore don't understand why take into account rate at all (#6 and #8)
                                                                                  • The numerator here seems to be trying to get all non-idle usage and does so by getting the full sum and subtracting the idle time. But why does one use count and the other sum?
                                                                                  • If we grab cpu seconds by filtering by mode=idle, then does adding the by (mode) add anything? There is only one mode anyways? My understanding of by (something) is more relevant when there are multiple values and we group the values by that category (as we do by cpu in this query)
                                                                                  • Lastly, as mentioned in bold above, what is with the double count(), in the numerator and denominator?

                                                                                  ANSWER

                                                                                  Answered 2022-Mar-19 at 12:37

                                                                                  Both of these count functions return the amount of CPU cores. If you take them out of this long query and execute, it'll immediately make sense:

                                                                                  count by (cpu) (node_cpu_seconds_total{instance="foo:9100"})
                                                                                  
                                                                                  # result:
                                                                                  {cpu="0"} 8
                                                                                  {cpu="1"} 8
                                                                                  

                                                                                  By putting the above into another count() function, you will get a value of 2, because there are just 2 metrics in the dataset. At this point, we can simplify the original query to this:

                                                                                  (
                                                                                    NUM_CPU
                                                                                    -
                                                                                    avg(
                                                                                      sum by(mode) (
                                                                                        rate(node_cpu_seconds_total{mode="idle",instance="foo:9100"}[1m])
                                                                                      )
                                                                                    )
                                                                                    * 100
                                                                                  )
                                                                                  / NUM_CPU
                                                                                  

                                                                                  The rest, however, is somewhat complicated. This:

                                                                                      sum by(mode) (
                                                                                        rate(node_cpu_seconds_total{mode="idle",instance="foo:9100"}[1m])
                                                                                      )
                                                                                  

                                                                                  ... is essentially the sum of idle time of all CPU cores (I'm intentionally skipping the context of time to make it simpler). It's not clear why there is by (mode), since the rate function inside has a filter, which makes it possible for only idle mode to appear. With or without by (mode) it returns just one value:

                                                                                  # with by (mode)
                                                                                  {mode="idle"} 0.99
                                                                                  
                                                                                  # without
                                                                                  {} 0.99
                                                                                  

                                                                                  avg() on top of that makes no sense at all. I assume, that the intention was to get the amount of idle time per CPU (by (cpu), that is). In this case it starts to make sense, although it is still unnecessary complex. Thus, at this point we can simplify the query to this:

                                                                                  (NUM_CPU - IDLE_TIME_TOTAL * 100) / NUM_CPU
                                                                                  

                                                                                  I don't know why it is so complicated, you can get the same result with a simple query like this:

                                                                                  100 * (1 - avg(rate(node_cpu_seconds_total{mode="idle", instance="foo:9100"}[1m])))
                                                                                  

                                                                                  Source https://stackoverflow.com/questions/71529645

                                                                                  QUESTION

                                                                                  Thanos-Query/Query-Frontend does not show any metrics
                                                                                  Asked 2022-Feb-24 at 15:46

                                                                                  Basically, I had installed Prometheues-Grafana from the kube-prometheus-stack using the provided helm chart repo prometheus-community

                                                                                  # helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
                                                                                  # helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack
                                                                                  

                                                                                  They are working fine.

                                                                                  But the problem I am facing now is integrating Thanos with this existing kube-prometheus-stack.

                                                                                  I installed thanos from the bitnami helm chart repo

                                                                                  # helm repo add bitnami https://charts.bitnami.com/bitnami
                                                                                  # helm install thanos bitnami/thanos
                                                                                  

                                                                                  I can load the Thanos Query Frontend GUI, but no metrics showing there.

                                                                                  I am struggling now to get it worked properly. Is it because of Thanos from a completely different helm chart and Prometheus-operator-grafana stack from another helm chart ?.

                                                                                  My Kubernetes cluster on AWS has been created using Kops. And, I use Gitlab pipeline and helm to deploy apps to the cluster.

                                                                                  ANSWER

                                                                                  Answered 2022-Feb-24 at 15:46

                                                                                  It's not enough to simply install them, you need to integrate prometheus with thanos.

                                                                                  Below I'll describe all steps you need to perform to get the result.

                                                                                  First short theory. The most common approach to integrate them is to use thanos sidecar container for prometheus pod. You can read more here.

                                                                                  How this is done:

                                                                                  (considering that installation is clean, it can be easily deleted and reinstalled from the scratch).

                                                                                  1. Get thanos sidecar added to the prometheus pod.

                                                                                  Pull kube-prometheus-stack chart:

                                                                                  $ helm pull prometheus-community/kube-prometheus-stack --untar
                                                                                  

                                                                                  You will have a folder with a chart. You need to modify values.yaml, two parts to be precise:

                                                                                  # Enable thanosService
                                                                                  prometheus:
                                                                                    thanosService:
                                                                                      enabled: true # by default it's set to false
                                                                                  
                                                                                  # Add spec for thanos sidecar
                                                                                  prometheus:
                                                                                    prometheusSpec:
                                                                                      thanos:
                                                                                        image: "quay.io/thanos/thanos:v0.24.0"
                                                                                        version: "v0.24.0"
                                                                                  

                                                                                  Keep in mind, this feature is still experimental:

                                                                                  ## This section is experimental, it may change significantly without deprecation notice in any release.
                                                                                  ## This is experimental and may change significantly without backward compatibility in any release.
                                                                                  ## ref: https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#thanosspec
                                                                                  

                                                                                  Once it's done, install the prometheus chart with edited values.yaml:

                                                                                  $ helm install prometheus . -n prometheus --create-namespace # installed in prometheus namespace
                                                                                  

                                                                                  And check that sidecar is deployed as well:

                                                                                  $ kubectl get pods -n prometheus | grep prometheus-0
                                                                                  prometheus-prometheus-kube-prometheus-prometheus-0       3/3     Running   0          67s
                                                                                  

                                                                                  It should be 3 containers running (by default it's 2). You can inspect it in more details with kubectl describe command.

                                                                                  1. Setup thanos chart and deploy it.

                                                                                  Pull the thanos chart:

                                                                                  $ helm pull bitnami/thanos --untar
                                                                                  

                                                                                  Edit values.yaml:

                                                                                  query:
                                                                                    dnsDiscovery:
                                                                                      enabled: true
                                                                                      sidecarsService: "prometheus-kube-prometheus-thanos-discovery" # service which was created before
                                                                                      sidecarsNamespace: "prometheus" # namespace where prometheus is deployed
                                                                                  

                                                                                  Save and install this chart with edited values.yaml:

                                                                                  $ helm install thanos . -n thanos --create-namespace
                                                                                  

                                                                                  Check that it works:

                                                                                  $ kubectl logs thanos-query-xxxxxxxxx-yyyyy -n thanos
                                                                                  

                                                                                  We are interested in this line:

                                                                                  level=info ts=2022-02-24T15:32:41.418475238Z caller=endpointset.go:349 component=endpointset msg="adding new sidecar with [storeAPI rulesAPI exemplarsAPI targetsAPI MetricMetadataAPI]" address=10.44.1.213:10901 extLset="{prometheus=\"prometheus/prometheus-kube-prometheus-prometheus\", prometheus_replica=\"prometheus-prometheus-kube-prometheus-prometheus-0\"}"
                                                                                  
                                                                                  1. Now go to the UI and see that metrics are available:

                                                                                  Good article to read:

                                                                                  Source https://stackoverflow.com/questions/71243202

                                                                                  QUESTION

                                                                                  Add Kubernetes scrape target to Prometheus instance that is NOT in Kubernetes
                                                                                  Asked 2022-Feb-13 at 20:24

                                                                                  I run prometheus locally as http://localhost:9090/targets with

                                                                                  docker run --name prometheus -d -p 127.0.0.1:9090:9090 prom/prometheus
                                                                                  

                                                                                  and want to connect it to several Kubernetes (cluster) instances we have. See that scraping works, try Grafana dashboards etc.

                                                                                  And then I'll do the same on dedicated server that will be specially for monitoring. However all googling gives me all different ways to configure prometheus that is already within one Kubernetes instance, and no way to read metrics from external Kubernetes.

                                                                                  How to add Kubernetes scrape target to Prometheus instance that is NOT in Kubernetes?

                                                                                  I have read Where Kubernetes metrics come from and checked that my (first) Kubernetes cluster has the Metrics Server.

                                                                                  kubectl get pods --all-namespaces | grep metrics-server 
                                                                                  

                                                                                  There is definitely no sense to add Prometheus instance into every Kubernetes (cluster) instance. One Prometheus must be able to read metrics from many Kubernetes clusters and every node within them.

                                                                                  P.S. Some old question has answer to install Prometheus in every Kubernetes and then use federation, that is just opposite from what I am looking for.

                                                                                  P.P.S. It is also strange for me, why Kubernetes and Prometheus that are #1 and #2 projects from Cloud Native Foundation don't have simple "add Kubernetes target in Prometheus" button or simple step.

                                                                                  ANSWER

                                                                                  Answered 2021-Dec-28 at 08:33

                                                                                  There are many agents capable of saving metrics collected in k8s to remote Prometheus server outside the cluster, example Prometheus itself now support agent mode, exporter from Opentelemetry, or using managed Prometheus etc.

                                                                                  Source https://stackoverflow.com/questions/70457308

                                                                                  QUESTION

                                                                                  Sucessfully queries the azure monitor service. Workspace not found. While using azuremarket place Grafana
                                                                                  Asked 2022-Jan-13 at 15:51

                                                                                  I'm trying to use azure monitor as a data source for grafana. The grafana server was created from Azure Marketplace. I used Service Principal for authentication and while clicking the 'save and test' button, I get the following error"

                                                                                  ' 1. Successfully queried the Azure Monitor service. 2. Workspace not found. '

                                                                                  Can you please help me with this issue? Thank you.

                                                                                  ANSWER

                                                                                  Answered 2021-Dec-21 at 13:49

                                                                                  Adding a Azure App Insights resource to the monitored Subscription solved the problem. On this step the first Monitoring Workspace for the Subscription was created. On an older Resource I had to migrate to Workspace-based Application Insights to fix the error. It seams Grafana only works with the new Workspace-based Application Insights resources

                                                                                  Source https://stackoverflow.com/questions/70159980

                                                                                  QUESTION

                                                                                  Grafana - Is it possible to use variables in Loki-based dashboard query?
                                                                                  Asked 2022-Jan-07 at 12:41

                                                                                  I am working on a Loki-based Dashboard on Grafana. I have one panel for searching text in the Loki trace logs, the current query is like:

                                                                                  {job="abc-service"}
                                                                                  |~ "searchTrace"
                                                                                  |json
                                                                                  |line_format "{if .trace_message}} Message: \t{{.trace_message}} {{end}}"
                                                                                  

                                                                                  Where searchTrace is a variable of type "Text box" for the user to input search text.

                                                                                  I want to include another variable skipTestLog to skip logs created by some test cron tasks. skipTestLog is a custom variable of two options: Yes,No.

                                                                                  Suppose the logs created by test cron tasks contain the text CronTest in the field trace_message after the json parser, are there any ways to filter them out based on the selected value of skipTestLog?

                                                                                  ANSWER

                                                                                  Answered 2022-Jan-07 at 12:41

                                                                                  Create a key/value custom variable like in the following example:

                                                                                  Use the variable like in the following example:

                                                                                  Source https://stackoverflow.com/questions/70616495

                                                                                  QUESTION

                                                                                  PostgreSQL Default Result Limit
                                                                                  Asked 2022-Jan-01 at 15:34

                                                                                  I'm using Grafana and PostgreSQL 13 for visualizing. There are many users in the Grafana and they could send queries to their own databases.

                                                                                  I need to set a default result limit for sent queries. (Like 1000) But I couldn't find a solution. I analyzed the PgPool to rewrite the query but I think it couldn't do that.

                                                                                  Is there any solution for that? I'm not sure but maybe I need a TCP Proxy which can do.

                                                                                  ANSWER

                                                                                  Answered 2022-Jan-01 at 15:34

                                                                                  The most popular solution, as far as I know, is PgBouncer. PgBouncer is a lightweight connection pooler for PostgreSQL. It acts as a Postgres server, so simply point your Grafana and other clients to the PgBouncer port.

                                                                                  Here are some installation guides for Linux (Ubuntu, Debian, CentOS):

                                                                                  Source https://stackoverflow.com/questions/70549256

                                                                                  QUESTION

                                                                                  Trigger Beam ParDo at window closing only
                                                                                  Asked 2021-Dec-15 at 18:24

                                                                                  I have a pipeline that read events from Kafka. I want to count and log the event count only when the window closes. By doing this I will only have one output log per Kafka partition/shard on each window. I use a timestamp in the header which I truncate to the hour to create a collection of hourly timestamps. I group the timestamps by hour and I log the hourly timestamp and count. This log will be sent to Grafana to create a dashboard with the counts.

                                                                                  Below is how I fetch the data from Kafka and where it defines the window duration:

                                                                                  int windowDuration = 5;
                                                                                  p.apply("Read from Kafka",KafkaIO.read()
                                                                                              .withBootstrapServers(options.getSourceBrokers().get())
                                                                                              .withTopics(srcTopics)
                                                                                              .withKeyDeserializer(ByteArrayDeserializer.class)
                                                                                              .withValueDeserializer(ConfluentSchemaRegistryDeserializerProvider
                                                                                              .of(options.getSchemaRegistryUrl().get(), options.getSubject().get()))
                                                                                                      .commitOffsetsInFinalize())
                                                                                    .apply("Windowing of " + windowDuration +" seconds" , 
                                                                                              Window.>into(
                                                                                              FixedWindows.of(Duration.standardSeconds(windowDuration))));
                                                                                  

                                                                                  The next step in the pipeline is to produce two collections from the above collection one with the events as GenericRecord and the other with the hourly timestamp, see below. I want a trigger (I believe) to be applied only two the collection holding the counts. So that it only prints the count once per window. Currently as is, it prints a count every time it reads from Kafka creating a large number of entries.

                                                                                    tuplePCollection.get(createdOnTupleTag)
                                                                                    .apply(Count.perElement())
                                                                                    .apply( MapElements.into(TypeDescriptors.strings())
                                                                                    .via( (KV recordCount) -> recordCount.getKey() +
                                                                                      ": " + recordCount.getValue()))
                                                                                    .apply( ParDo.of(new LoggerFn.logRecords()));
                                                                                  

                                                                                  Here is the DoFn I use to log the counts:

                                                                                   class LoggerFn extends DoFn {
                                                                                          @ProcessElement
                                                                                          public void process(ProcessContext c) {
                                                                                              T e = (T)c.element();
                                                                                              LOGGER.info(e);
                                                                                              c.output(e);
                                                                                          }
                                                                                      }
                                                                                  

                                                                                  ANSWER

                                                                                  Answered 2021-Dec-15 at 18:24

                                                                                  You can use the trigger “Window.ClosingBehavior”. You need to specify under which conditions a final pane will be created when a window is permanently closed. You can use these options:

                                                                                  • FIRE_ALWAYS: Always Fire the last Pane.

                                                                                  • FIRE_IF_NON_EMPTY: Only Fire the last pane if there is new data since previous firing.

                                                                                  You can see this example.

                                                                                  // We first specify to never emit any panes
                                                                                   .triggering(Never.ever())
                                                                                   
                                                                                   // We then specify to fire always when closing the window. This will emit a
                                                                                   // single final pane at the end of allowedLateness
                                                                                   .withAllowedLateness(allowedLateness, Window.ClosingBehavior.FIRE_ALWAYS)
                                                                                   .discardingFiredPanes())
                                                                                  

                                                                                  You can see more information about this trigger.

                                                                                  Source https://stackoverflow.com/questions/70351827

                                                                                  Community Discussions, Code Snippets contain sources that include Stack Exchange Network

                                                                                  Vulnerabilities

                                                                                  No vulnerabilities reported

                                                                                  Install grafana

                                                                                  Unsure if Grafana is for you? Watch Grafana in action on play.grafana.org!.
                                                                                  Get Grafana
                                                                                  Installation guides

                                                                                  Support

                                                                                  The Grafana documentation is available at grafana.com/docs.
                                                                                  Find more information at:
                                                                                  Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
                                                                                  Find more libraries
                                                                                  Explore Kits - Develop, implement, customize Projects, Custom Functions and Applications with kandi kits​
                                                                                  Save this library and start creating your kit

                                                                                  Share this Page

                                                                                  share link

                                                                                  Consider Popular Dashboard Libraries

                                                                                  grafana

                                                                                  by grafana

                                                                                  AdminLTE

                                                                                  by ColorlibHQ

                                                                                  ngx-admin

                                                                                  by akveo

                                                                                  kibana

                                                                                  by elastic

                                                                                  appsmith

                                                                                  by appsmithorg

                                                                                  Try Top Libraries by grafana

                                                                                  k6

                                                                                  by grafanaGo

                                                                                  loki

                                                                                  by grafanaGo

                                                                                  pyroscope

                                                                                  by grafanaGo

                                                                                  mimir

                                                                                  by grafanaGo

                                                                                  tempo

                                                                                  by grafanaGo

                                                                                  Compare Dashboard Libraries with Highest Support

                                                                                  grafana

                                                                                  by grafana

                                                                                  kibana

                                                                                  by elastic

                                                                                  AdminLTE

                                                                                  by ColorlibHQ

                                                                                  django-jet

                                                                                  by geex-arts

                                                                                  appsmith

                                                                                  by appsmithorg

                                                                                  Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
                                                                                  Find more libraries
                                                                                  Explore Kits - Develop, implement, customize Projects, Custom Functions and Applications with kandi kits​
                                                                                  Save this library and start creating your kit