kapacitor | Open source framework for processing monitoring | Time Series Database library
kandi X-RAY | kapacitor Summary
Support
Quality
Security
License
Reuse
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample Here
kapacitor Key Features
kapacitor Examples and Code Snippets
Trending Discussions on kapacitor
Trending Discussions on kapacitor
QUESTION
I'm getting started with Kapacitor and have been trying to run the first guide in the Kapacitor documentation, but with data I already have. I managed to define a task, but I can neither enable it nor can I run a backfill. I came across this question, which is similar to my problem, but the answer there didn't help. In contrast to the error message there I get empty strings for database, retention policy, and/or measurement.
In Kapacitor config I set an InfluxDB connection to the local host instance with the name localhost (which has a database mydb and the measurements weather.current.clouds and weather.current.visibility with default retention policy autogen) and created the following weathertest.tick script:
dbrp "mydb"."autogen"
var clouds = batch
|query('select mean(value) / 100.0 as val from "mydb"."autogen"."weather.current.clouds"')
.period(1h)
.every(1h)
.groupBy(time(1m), *)
.fill(0)
var vis = batch
|query('select mean(value) / 10000.0 as val from "mydb"."autogen"."weather.current.visibility"')
.period(1h)
.every(1h)
.groupBy(time(1m), *)
.fill(0)
clouds
|join(vis)
.as('c', 'v')
|eval(lambda: 100 * (1 - "c.val") * "v.val")
.as('pcent')
|influxDBOut()
.cluster('localhost')
.database('mydb')
.retentionPolicy('autogen')
.measurement('testmetric')
.tag('host', 'myhost.local')
.tag('key', 'weather.current.lightidx')
This is what I came up with after hours of trial and (especially) error. As given in the title, when I try to enable my task with kapacitor enable weathertest, I get the error message enabling task weathertest: batch query is not allowed to request data from ""."". Same thing when I try to record as in the "Backfill" example. Also, in that example there is a start and a stop date for limiting the time frame. The time format given there is wrong and is not understood by Kapacitor. Instead of e. g. 2015-10-01 I have to put in 2015-10-01T00:00Z to make it at least pass the error message regarding time format error.
In the Kapacitor logs there is not a single line regarding these errors, only when I try to remove a record, I get something like remove /var/lib/kapacitor/replay/1f5...750.brpl: no such file or directory and this can be found in the logs. There are lots of info lines in the logs showing successful POSTs to/from InfluxDB for the _internal database with HTTP response result 204.
Has anyone an Idea what I may be doing wrong?
ANSWER
Answered 2020-May-15 at 11:41OK, after the weekend I tried again. Without any change it accepted my script now in the failing steps, however, now I was able to find error messages in the log. The node mentioned there was the eval node and pointed towards a type mismatch. When I changed the line
|eval(lambda: 100 * (1 - "c.val") * "v.val")
to
|eval(lambda: 100.0 * (1.0 - "c.val") * "v.val")
the error messages were gone and the command kapacitor show weathertest showed a rather sane content now.
Furthermore, I redefined, recorded, replayed and deleted the tasks and recordings during my tests over and over again and I may have forgotten to redefine tasks after making changes to the tick script (not really sure). After changing the above, redefining the task and replaying it I finally found the expected data in the InfluxDB instance.
QUESTION
I need to perform some background advanced calculations on my data after it is collected in InfluxDb which is stored on the edge server, which means I have limited resources for the calculations. Also I cannot block the data collection while I do calculations. I am weighing using Kapacitor UDF streams vs custom Python scripts.
Please note I need to make the scripts configurable so that I can easily deply them to different environments with different sensors
ANSWER
Answered 2020-Mar-02 at 15:20It probably makes little difference, in general, especially for 'simple' usecases, though I lean towards standalone python scripts. (It may be better to use Kapacitor if you can cover your usecase using the kapacitor language for this instead of python based UDFs, but I found it insufficient since I needed to retrieve additional data from other databases)
- Standalone python scripts could be a bit lighter, since you don't need to run the Kapacitor service.
- Standalone python scripts could be a bit more configurable. Kapacitor is also pretty configurable but you'd have to spend a bit of time learning how to use it.
- Standalone python scripts could be a bit more stable. I've experimented with python UDF's a couple of years ago and found them unstable and buggy. While this may have improved by now, you'd still be relying on this being supported and if
gois not your language you might have trouble debugging and fixing issues yourself.
QUESTION
I am writing an app in pure Java without using Spring/SpringBoot or any other frameworks. I am using the TICK stack (Telegraf, Influx, Chronograf, and Kapacitor) as my metrics backend and viz platform. I am using Telegraf with Statsd plugin activated on localhost 8125. The problem is that my app is not sending any metrics to the system. However, I wrote a simple Spring Boot app which pushed the relevant metrics to my dashboard. Also, doing an echo "api.msgs.ok:10|c" | nc -C -w 1 -u localhost 8125 pushes the metrics to my dashboard as well.
The sample code I am trying to run is here:
import io.micrometer.statsd.StatsdConfig;
import io.micrometer.statsd.StatsdFlavor;
import io.micrometer.statsd.StatsdMeterRegistry;
import java.util.Properties;
public class TestMicrometer {
public static void main(String[] args) throws InterruptedException{
Properties properties = new Properties();
//properties.put("statsd.host","127.0.0.1");
//properties.put("statsd.port","8125");
StatsdConfig config = new StatsdConfig() {
//Will use localhost 8125 if get returns null always
@Override
public String get(String k) {
/*
System.out.println("Key " +k+ " is called");
if (k.equals("statsd.host")){
System.out.println("The host is "+properties.getProperty(k));
}
return properties.getProperty(k);
*/
return null;
}
@Override
public StatsdFlavor flavor() {
return StatsdFlavor.TELEGRAF;
}
};
MeterRegistry registry = new StatsdMeterRegistry(config, Clock.SYSTEM);
Counter.builder("loop.counter.test").description("TOTAL_LOOP_COUNTER").tags("LOOP.COUNTER","SAMPLE.LOOP.METER").register(registry).increment();
Metrics.addRegistry(registry);
for (int i = 0; i < 50; i++) {
Metrics.counter("loop.counter.test").increment();
}
}
}
Any help will be highly appreciated :)
ANSWER
Answered 2020-Jan-15 at 01:16Publishing of StatsD metrics is asynchronous, and this example simply terminates too quickly. Put a Thread.sleep(1000) at the end, and metrics are published fine.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install kapacitor
kapacitor – a CLI program for calling the Kapacitor API.
kapacitord – the Kapacitor server daemon.
This README gives you a high level overview of what Kapacitor is and what its like to use it. As well as some details of how it works. To get started using Kapacitor see this guide. After you finish the getting started exercise you can check out the TICKscripts for different Telegraf plugins.
Support
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesExplore Kits - Develop, implement, customize Projects, Custom Functions and Applications with kandi kits
Save this library and start creating your kit
Share this Page
