Big Data Libraries - Page 9

Riak client for Javascript

JavaScript 479Updated: 4 y ago License: No License (No License)

This project provides Apache Spark SQL, RDD, DataFrame and Dataset examples in Scala language

Scala 477Updated: 2 y ago License: No License (No License)

Spring Hadoop Samples

Java 476Updated: 4 y ago License: No License (No License)

PandaPy has the speed of NumPy and the usability of Pandas 10x to 50x faster (by @firmai)

Python 476Updated: 4 y ago License: No License (No License)

DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.

Java 476Updated: 2 y ago License: Permissive (Apache-2.0)

Java 475Updated: 4 y ago License: No License (No License)

Simplifying robust end-to-end machine learning on Apache Spark.

Scala 472Updated: 4 y ago License: Permissive (Apache-2.0)

SIMD-accelerated UTF-8 validation for Rust.

Rust 469Updated: 2 y ago License: Proprietary (Proprietary)

Examples for High Performance Spark

Scala 467Updated: 2 y ago License: Proprietary (Proprietary)

Personal Data Engineering Projects

Jupyter Notebook 467Updated: 2 y ago License: No License (No License)

Arctic is a streaming lake warehouse service open sourced by NetEase

Java 467Updated: 2 y ago License: Permissive (Apache-2.0)

Hybrid data integration service that simplifies ETL at scale

cloud_api 465Updated: Current License: Proprietary (Proprietary)

Spark Streaming+Flume+Kafka+HBase+Hadoop+Zookeeper实现实时日志分析统计；SpringBoot+Echarts实现数据可视化展示

Java 461Updated: 2 y ago License: No License (No License)

A Scala feature transformation library for data science and machine learning

Scala 460Updated: 2 y ago License: Permissive (Apache-2.0)

Stream Data Mining Library for Spark Streaming

Scala 460Updated: 4 y ago License: Permissive (Apache-2.0)

pyspark methods to enhance developer productivity 📣 👯 🎉

Python 455Updated: 2 y ago License: No License (No License)

Deploying complex solutions, magically.

Python 455Updated: 4 y ago License: Permissive (MIT)

Kafka Connect HDFS connector

Java 452Updated: 2 y ago License: Proprietary (Proprietary)

Presto Ethereum Connector -- SQL on Ethereum

Java 450Updated: 4 y ago License: Permissive (Apache-2.0)

Generic Data Ingestion & Dispersal Library for Hadoop

Java 449Updated: 3 y ago License: Proprietary (Proprietary)

Automated Repair Awesomeness for Apache Cassandra

Java 448Updated: 2 y ago License: Permissive (Apache-2.0)

This is a repo documenting the best practices in PySpark.

Jupyter Notebook 447Updated: 2 y ago License: No License (No License)

An Apache Flink subproject to provide storage for dynamic tables.

Java 445Updated: 2 y ago License: Permissive (Apache-2.0)

An open-source columnar data format designed for fast & realtime analytic with big data.

Java 443Updated: 3 y ago License: Permissive (Apache-2.0)

UNMAINTAINED - An API compatible open source server for interacting with devices speaking the spark-protocol

JavaScript 443Updated: 4 y ago License: Strong Copyleft (AGPL-3.0)

Distributed Stockfish analysis for lichess.org

Rust 440Updated: 3 y ago License: Proprietary (Proprietary)

Moonbox is a DVtaaS (Data Virtualization as a Service) Platform

JavaScript 438Updated: 4 y ago License: Proprietary (Proprietary)

Vectorized processing for Apache Arrow

C++ 438Updated: 4 y ago License: No License (No License)

The Vector Optimized Library of Kernels

C++ 436Updated: 2 y ago License: Weak Copyleft (LGPL-3.0)

a lightweight 3d particle engine in javascript, compatible with THREE.js and TWEEN.js

JavaScript 432Updated: 4 y ago License: No License (No License)

Diagrams describing Apache Hadoop internals (2.3.0 or later).

HTML 429Updated: 4 y ago License: No License (No License)

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.

Scala 426Updated: 3 y ago License: Permissive (Apache-2.0)

Scala examples for learning to use Spark

Scala 425Updated: 4 y ago License: Permissive (MIT)

Data Engineering Practice Problems

Python 425Updated: 3 y ago License: No License (No License)

Stanford CoreNLP wrapper for Apache Spark

Scala 423Updated: 4 y ago License: Strong Copyleft (GPL-3.0)

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

Java 421Updated: 2 y ago License: Permissive (Apache-2.0)

Prisma generator for automatically generating documentation reference from the Prisma schema.

TypeScript 419Updated: 2 y ago License: Permissive (MIT)

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.

Java 418Updated: 3 y ago License: Proprietary (Proprietary)

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Jupyter Notebook 417Updated: 3 y ago License: Permissive (MIT)

[MAINTENANCE ONLY] DataStax PHP Driver for Apache Cassandra

C 415Updated: 4 y ago License: Proprietary (Proprietary)

Ansible playbook that installs a Hadoop cluster, with HBase, Hive, Presto for analytics, and Ganglia, Smokeping, Fluentd, Elasticsearch and Kibana for monitoring and centralized log indexing.

Shell 415Updated: 4 y ago License: Permissive (Apache-2.0)

Utility tool to load Data into Cassandra to help you writing good isolated JUnit Test into your application

Java 414Updated: 4 y ago License: Weak Copyleft (LGPL-3.0)

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

Java 412Updated: 2 y ago License: Permissive (Apache-2.0)

Mirror of Apache Eagle

Java 410Updated: 4 y ago License: Permissive (Apache-2.0)

Iceberg is a table format for large, slow-moving tabular data

Java 409Updated: 3 y ago License: Permissive (Apache-2.0)

Mirror of Apache Eagle

Java 409Updated: 4 y ago License: Permissive (Apache-2.0)

Apache Tez

Java 408Updated: 2 y ago License: Permissive (Apache-2.0)

24MHz sampling rate Logic Analyzer based on fx2lafw

C 408Updated: 2 y ago License: No License (No License)

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

Scala 408Updated: 2 y ago License: Permissive (Apache-2.0)

Stroom is a highly scalable data storage, processing and analysis platform.

Java 406Updated: 2 y ago License: Permissive (Apache-2.0)