Kafka Libraries - Page 4

Sync data between persistence engines, like ETL only not stodgy

Go 1408Updated: 1 y ago License: Permissive (BSD-3-Clause)

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Scala 1400Updated: 1 y ago License: Proprietary (Proprietary)

Kafka (and Zookeeper) in Docker

Shell 1396Updated: 1 y ago License: Permissive (Apache-2.0)

Distributed Prometheus time series database

Scala 1378Updated: 1 y ago License: Permissive (Apache-2.0)

Collect, aggregate, and visualize a data ecosystem's metadata

Java 1360Updated: 10 mo ago License: Permissive (Apache-2.0)

HiBench is a big data benchmark suite.

Java 1351Updated: 1 y ago License: Proprietary (Proprietary)

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

Shell 1332Updated: 1 y ago License: Permissive (Apache-2.0)

TBase is an enterprise-level distributed HTAP database. Through a single database cluster to provide users with highly consistent distributed database services and high-performance data warehouse services, a set of integrated enterprise-level solutions is formed.

C 1321Updated: 11 mo ago License: Proprietary (Proprietary)

This library provides implementations of many algorithms and data structures that are useful for bioinformatics. All provided implementations are rigorously tested via continuous integration.

Rust 1311Updated: 10 mo ago License: Permissive (MIT)

A system design tool that allows you to simulate data flow of distributed systems.

TypeScript 1305Updated: 1 y ago License: Strong Copyleft (GPL-3.0)

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

Java 1302Updated: 1 y ago License: Permissive (Apache-2.0)

Open source specification and implementation of Knative event binding and delivery

Go 1292Updated: 10 mo ago License: Permissive (Apache-2.0)

DDMQ is a distributed messaging product with low latency, high throughput and high availability.

Java 1257Updated: 1 y ago License: Permissive (Apache-2.0)

Kazoo is a high-level Python library that makes it easier to use Apache Zookeeper.

Python 1248Updated: 1 y ago License: Permissive (Apache-2.0)

A Ruby client library for Apache Kafka

Ruby 1238Updated: 11 mo ago License: Permissive (Apache-2.0)

A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka

Rust 1235Updated: 10 mo ago License: Permissive (MIT)

EventMesh is a new generation serverless event middleware for building distributed event-driven applications.

Java 1199Updated: 1 y ago License: Permissive (Apache-2.0)

Examples for running Debezium (Configuration, Docker Compose files etc.)

JavaScript 1196Updated: 1 y ago License: Permissive (Apache-2.0)

Example project implementing best practices for PySpark ETL jobs and applications.

Python 1195Updated: 10 mo ago License: No License (No License)

Test your code without writing mocks with ephemeral Docker containers 📦 Setup popular services with just a couple lines of code ⏱️ No bash, no yaml, only code 💻

Go 1188Updated: 10 mo ago License: Permissive (MIT)

High performance mqtt broker

Go 1186Updated: 11 mo ago License: Permissive (Apache-2.0)

KillrWeather is a reference application (work in progress) showing how to easily integrate streaming and batch data processing with Apache Spark Streaming, Apache Cassandra, Apache Kafka and Akka for fast, streaming computations on time series data in asynchronous event-driven environments.

Scala 1185Updated: 1 y ago License: Permissive (Apache-2.0)

Scalable, fault-tolerant application-layer sharding for Node.js applications

JavaScript 1177Updated: 1 y ago License: Permissive (MIT)

分布式系统服务ZooKeeper的学习历程

Java 1161Updated: 10 mo ago License: No License (No License)

Vendor-neutral programmable observability pipelines.

Go 1152Updated: 10 mo ago License: Permissive (Apache-2.0)

50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak

Shell 1147Updated: 10 mo ago License: Permissive (MIT)

StreamSets Data Collector - Continuous big data and cloud platform ingest infrastructure

Java 1145Updated: 3 y ago License: Permissive (Apache-2.0)

[DEPRECATED] Docker images for Confluent Platform.

Python 1127Updated: 10 mo ago License: Permissive (Apache-2.0)

java-study 是本人学习Java过程中记录的一些代码！从Java基础的数据类型、jdk1.8的Lambda、Stream和日期的使用、 IO流、数据集合、多线程使用、并发编程、23种设计模式示例代码、常用的工具类， 以及一些常用框架，netty、mina、springboot、kafka、storm、zookeeper、redis、elasticsearch、hbase、hive等等。

Java 1117Updated: 10 mo ago License: Permissive (Apache-2.0)

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Python 1114Updated: 1 y ago License: Permissive (MIT)

Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance.

Python 1111Updated: 1 y ago License: Permissive (Apache-2.0)

TestDisk & PhotoRec

C 1108Updated: 10 mo ago License: Strong Copyleft (GPL-2.0)

450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

Python 1101Updated: 10 mo ago License: Proprietary (Proprietary)

Authenticate socket.io incoming connections with JWTs

JavaScript 1101Updated: 3 y ago License: Permissive (MIT)

franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.4+. Producing, consuming, transacting, administrating, etc.

Go 1101Updated: 10 mo ago License: Permissive (BSD-3-Clause)

Python Stream Processing. A Faust fork

Python 1059Updated: 10 mo ago License: Proprietary (Proprietary)

Distributed Stream and Batch Processing

Java 1054Updated: 1 y ago License: Proprietary (Proprietary)

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

Java 1041Updated: 1 y ago License: Permissive (Apache-2.0)

A Meteor package that extends Mongo.Collection to provide support for specifying a schema and then validating against that schema when inserting and updating.

JavaScript 1030Updated: 1 y ago License: Permissive (MIT)

C/C++ library for processing configuration files

C 1001Updated: 12 mo ago License: Weak Copyleft (LGPL-2.1)

Kubernetes Universal Declarative Operator (KUDO)

Go 997Updated: 2 y ago License: Permissive (Apache-2.0)

MapReduce, Spark, Java, and Scala for Data Algorithms Book

Java 996Updated: 2 y ago License: Proprietary (Proprietary)

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes

Java 992Updated: 10 mo ago License: Permissive (Apache-2.0)

an extension to the sidekiq message processing to track your jobs

Ruby 992Updated: 1 y ago License: Permissive (MIT)

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 [DEPRECATED]

Go 979Updated: 3 y ago License: Permissive (MIT)

This repo contains a sample application based on a Garage Management System for Pitstop - a fictitious garage. The primary goal of this sample is to demonstrate several software-architecture concepts like: Microservices, CQRS, Event Sourcing, Domain Driven Design (DDD), Eventual Consistency.

JavaScript 977Updated: 11 mo ago License: Permissive (Apache-2.0)

Rust client for Apache Kafka

Rust 965Updated: 11 mo ago License: Permissive (MIT)

WE HAVE MOVED to Apache Incubator. https://cwiki.apache.org/FLUME/ . Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. The system is centrally managed and allows for intelligent dynamic management. It uses a simple extensible data model that allows for online analytic applications.

Java 941Updated: 1 y ago License: Permissive (Apache-2.0)

Wormhole is a SPaaS (Stream Processing as a Service) Platform

JavaScript 937Updated: 2 y ago License: Proprietary (Proprietary)

Streaming reference architecture for ETL with Kafka and Kafka-Connect. You can find more on http://lenses.io on how we provide a unified solution to manage your connectors, most advanced SQL engine for Kafka and Kafka Streams, cluster monitoring and alerting, and more.

Scala 937Updated: 10 mo ago License: Permissive (Apache-2.0)