Big Data Libraries - Page 10

Apache Parquet

C++ 406Updated: 2 y ago License: Permissive (Apache-2.0)

Unicorn for node.js

JavaScript 404Updated: 4 y ago License: Permissive (MIT)

MIPP is a portable wrapper for SIMD instructions written in C++11. It supports NEON, SSE, AVX and AVX-512.

C++ 402Updated: 2 y ago License: Permissive (MIT)

Web tool for Kafka Connect |

JavaScript 400Updated: 4 y ago License: Proprietary (Proprietary)

大数据实践项目 Hadoop、Spark、Kafka、Hbase、Flink.....

Java 398Updated: 2 y ago License: Permissive (Apache-2.0)

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

Jupyter Notebook 397Updated: 2 y ago License: Permissive (MIT)

Stream computing platform for bigdata

Java 396Updated: 3 y ago License: Permissive (Apache-2.0)

Apache Spark training material

Scala 394Updated: 4 y ago License: No License (No License)

ODPS Python SDK and data analysis framework

Python 390Updated: 2 y ago License: Permissive (Apache-2.0)

This repo presents a reference architecture for running serverless MapReduce jobs. This has been implemented using AWS Lambda and Amazon S3.

JavaScript 388Updated: 4 y ago License: Proprietary (Proprietary)

Apache Flink Playgrounds

Java 385Updated: 2 y ago License: Permissive (Apache-2.0)

DataStax C/C++ Driver for Apache Cassandra

C++ 384Updated: 2 y ago License: Permissive (Apache-2.0)

Read-only mirror of ASF Git Repo for jclouds

Java 383Updated: 4 y ago License: No License (No License)

SIMD-accelerated base64 codecs

C 383Updated: 2 y ago License: Proprietary (Proprietary)

Stock inference engine using Spring XD, Apache Geode / GemFire and Spark ML Lib.

Java 381Updated: 2 y ago License: Permissive (Apache-2.0)

PySpark test helper methods with beautiful error messages

Python 379Updated: 2 y ago License: Permissive (MIT)

Compute over Data framework for public, transparent, and optionally verifiable computation

Go 378Updated: 2 y ago License: Permissive (Apache-2.0)

Oozie - workflow engine for Hadoop

Java 376Updated: 4 y ago License: Permissive (Apache-2.0)

Mazerunner extends a Neo4j graph database to run scheduled big data graph compute algorithms at scale with HDFS and Apache Spark.

Java 376Updated: 4 y ago License: Permissive (Apache-2.0)

A lightweight IoT edge analytics software

Go 375Updated: 4 y ago License: Permissive (Apache-2.0)

Scripts used to setup a Spark cluster on EC2

Python 374Updated: 4 y ago License: Permissive (Apache-2.0)

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

Kotlin 374Updated: 2 y ago License: Permissive (Apache-2.0)

An ad hoc query service based on the spark sql engine.(基于spark sql引擎的即席查询服务)

JavaScript 373Updated: 3 y ago License: Permissive (MIT)

Repository holding configuration files for running an HDFS cluster in Kubernetes

Shell 372Updated: 2 y ago License: Permissive (Apache-2.0)

the data and ipython notebook of my attempt to solve the kaggle titanic problem

Jupyter Notebook 371Updated: 3 y ago License: No License (No License)

TerichDB, an open source data store based on terark engine

C++ 368Updated: 4 y ago License: Strong Copyleft (AGPL-3.0)

This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

Java 367Updated: 2 y ago License: Permissive (Apache-2.0)

A Spark plugin for reading and writing Excel files

Scala 366Updated: 2 y ago License: Permissive (Apache-2.0)

This is a tool which used to manage and monitor ClickHouse database

Go 366Updated: 2 y ago License: Permissive (Apache-2.0)

Performance tests for Apache Spark

Scala 366Updated: 4 y ago License: Permissive (Apache-2.0)

🏐 Apache Parquet for modern .NET

C# 361Updated: 2 y ago License: Permissive (MIT)

Optimized Analytics Package for Spark* Platform

Scala 359Updated: 4 y ago License: Permissive (Apache-2.0)

Information for setting up for the BerkeleyX Spark Intro MOOC, and lab assignments for the course

Python 356Updated: 4 y ago License: No License (No License)

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

Java 355Updated: 2 y ago License: No License (No License)

🧭 Use Prisma as a multi-tenant provider for your application

TypeScript 354Updated: 2 y ago License: Permissive (MIT)

Includes notes on Apache Spark, Spark for Physics, Jupyter notebook examples for Spark, Oracle and other DB systems.

Jupyter Notebook 354Updated: 2 y ago License: Permissive (Apache-2.0)

Provides support to increase developer productivity in Java when using Apache Cassandra. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.

Java 352Updated: 2 y ago License: Permissive (Apache-2.0)

Rails app for tracking trends in server logs - powered by the Cloudera Hadoop Distribution on EC2

Ruby 352Updated: 4 y ago License: No License (No License)

Real Time Big Data / IoT Machine Learning (Model Training and Inference) with HiveMQ (MQTT), TensorFlow IO and Apache Kafka - no additional data store like S3, HDFS or Spark required

Jupyter Notebook 352Updated: 2 y ago License: Permissive (Apache-2.0)

t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark

Python 350Updated: 2 y ago License: Permissive (MIT)

Stream Processing with Apache Flink - Scala Examples

Scala 349Updated: 2 y ago License: Permissive (Apache-2.0)

Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.

Rust 349Updated: 2 y ago License: Permissive (Apache-2.0)

Ytk-learn is a distributed machine learning library which implements most of popular machine learning algorithms(GBDT, GBRT, Mixture Logistic Regression, Gradient Boosting Soft Tree, Factorization Machines, Field-aware Factorization Machines, Logistic Regression, Softmax).

Java 348Updated: 3 y ago License: Permissive (MIT)

Node.js API for Apache Spark with Remote Client

JavaScript 348Updated: 4 y ago License: Permissive (Apache-2.0)

hadoop-common-2.2.0/bin

Shell 347Updated: 4 y ago License: No License (No License)

Former GraphX development repository. GraphX has been merged into Apache Spark; please submit pull requests there.

Scala 345Updated: 3 y ago License: Permissive (Apache-2.0)

Mirror of Apache Apex core

Java 344Updated: 4 y ago License: Permissive (Apache-2.0)

Enhanced PostgreSQL logical replication

Go 341Updated: 4 y ago License: Permissive (Apache-2.0)

My Insight Data Engineering Fellowship project. I implemented a big data processing pipeline based on ​lambda architecture​, that aggregates Twitter and US stock market data for user sentiment analysis using open source tools - ​Apache Kafka ​for data ingestions, Apache Spark ​& ​Spark Streaming ​for batch & real-time processing, ​Apache Cassandra f​ or storage, ​Flask​, ​Bootstrap and ​HighCharts f​ or frontend.

Scala 341Updated: 2 y ago License: No License (No License)

CDP Public Cloud is an integrated analytics and data management platform deployed on cloud services. It offers broad data analytics and artificial intelligence functionality along with secure user access and data governance features.

Java 340Updated: 2 y ago License: Permissive (Apache-2.0)