Big Data Libraries - Page 4

winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

Shell 1481Updated: 11 mo ago License: No License (No License)

A curated list of awesome Apache Spark packages and resources.

Shell 1467Updated: 12 mo ago License: Permissive (CC0-1.0)

A cluster computing framework for processing large-scale geospatial data

Java 1457Updated: 11 mo ago License: Permissive (Apache-2.0)

MLeap: Deploy ML Pipelines to Production

Scala 1449Updated: 1 y ago License: Permissive (Apache-2.0)

Java 1444Updated: 12 mo ago License: Permissive (Apache-2.0)

Fast, modern C++ DSP framework, FFT, Sample Rate Conversion, FIR/IIR/Biquad Filters (SSE, AVX, AVX-512, ARM NEON)

C++ 1425Updated: 11 mo ago License: Strong Copyleft (GPL-2.0)

Lightning-fast cluster computing in Java, Scala and Python.

Scala 1423Updated: 1 y ago License: No License (No License)

Base classes to use when writing tests with Spark

Scala 1414Updated: 1 y ago License: Permissive (Apache-2.0)

Kolo(former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.

JavaScript 1400Updated: 2 y ago License: Permissive (Apache-2.0)

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Python 1383Updated: 11 mo ago License: Permissive (Apache-2.0)

Write any function in minutes – whether to run a simple job that cleans up a database or build a more complex architecture. Creating functions is easier than ever before, whatever your chosen OS, platform, or development method.

cloud_api 1382Updated: Current License: Proprietary (Proprietary)

BitSail is a distributed high-performance data integration engine which supports batch, streaming and incremental scenarios. BitSail is widely used to synchronize hundreds of trillions of data every day.

Java 1363Updated: 12 mo ago License: Permissive (Apache-2.0)

High performance data store solution

Scala 1359Updated: 1 y ago License: Permissive (Apache-2.0)

Apache Parquet

Java 1357Updated: 12 mo ago License: Permissive (Apache-2.0)

HiBench is a big data benchmark suite.

Java 1351Updated: 1 y ago License: Proprietary (Proprietary)

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

Scala 1343Updated: 1 y ago License: Permissive (Apache-2.0)

An open optimized software library project for the ARM® Architecture

C 1340Updated: 1 y ago License: Proprietary (Proprietary)

DataStax Python Driver for Apache Cassandra

Python 1335Updated: 12 mo ago License: Permissive (Apache-2.0)

TBase is an enterprise-level distributed HTAP database. Through a single database cluster to provide users with highly consistent distributed database services and high-performance data warehouse services, a set of integrated enterprise-level solutions is formed.

C 1321Updated: 12 mo ago License: Proprietary (Proprietary)

Fluid, elastic data abstraction and acceleration for BigData/AI applications in cloud. (Project under CNCF)

Go 1320Updated: 11 mo ago License: Permissive (Apache-2.0)

LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.

Scala 1303Updated: 12 mo ago License: Permissive (Apache-2.0)

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

Java 1302Updated: 1 y ago License: Permissive (Apache-2.0)

A cluster computing framework for processing large-scale geospatial data

Java 1302Updated: 1 y ago License: Permissive (Apache-2.0)

GeoMesa is a suite of tools for working with big geo-spatial data in a distributed fashion.

Scala 1302Updated: 1 y ago License: Permissive (Apache-2.0)

SIMD Vector Classes for C++

C++ 1302Updated: 1 y ago License: Permissive (BSD-3-Clause)

LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.

Scala 1298Updated: 12 mo ago License: Permissive (Apache-2.0)

Distributed deep learning on Hadoop and Spark clusters.

Jupyter Notebook 1265Updated: 1 y ago License: Permissive (Apache-2.0)

cluster data collected from production clusters in Alibaba for cluster management research

Jupyter Notebook 1256Updated: 12 mo ago License: No License (No License)

🔥🔥 BigData 💥 大数据 💥大数据AllData平台，通过二开大数据BigData生态组件，以及大数据BigData采集、大数据BigData存储、大数据BigData计算、大数据BigData开发来建设开源社区大数据BigData平台。联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

Java 1236Updated: 1 y ago License: Permissive (Apache-2.0)

SQL-based streaming analytics platform at scale

Java 1219Updated: 1 y ago License: Permissive (Apache-2.0)

fastest text uwuifier in the west

Rust 1215Updated: 12 mo ago License: Permissive (MIT)

Jupyter magics and kernels for working with remote Spark clusters

Python 1213Updated: 1 y ago License: Proprietary (Proprietary)

A script to easily create and destroy an Apache Cassandra cluster on localhost

Python 1202Updated: 12 mo ago License: Permissive (Apache-2.0)

Example project implementing best practices for PySpark ETL jobs and applications.

Python 1195Updated: 11 mo ago License: No License (No License)

DataStax Node.js Driver for Apache Cassandra

JavaScript 1192Updated: 1 y ago License: Permissive (Apache-2.0)

Dremio - the missing link in modern data

Java 1190Updated: 1 y ago License: Permissive (Apache-2.0)

KillrWeather is a reference application (work in progress) showing how to easily integrate streaming and batch data processing with Apache Spark Streaming, Apache Cassandra, Apache Kafka and Akka for fast, streaming computations on time series data in asynchronous event-driven environments.

Scala 1185Updated: 1 y ago License: Permissive (Apache-2.0)

Apache Spark 官方文档中文版

JavaScript 1184Updated: 12 mo ago License: Proprietary (Proprietary)

RisingWave: the next-generation streaming database in the cloud.

Rust 1183Updated: 2 y ago License: Permissive (Apache-2.0)

Scalable, fault-tolerant application-layer sharding for Node.js applications

JavaScript 1177Updated: 1 y ago License: Permissive (MIT)

A library for time series analysis on Apache Spark

Scala 1175Updated: 1 y ago License: Permissive (Apache-2.0)

Apache InLong - a one-stop integration framework for massive data

Java 1174Updated: 11 mo ago License: Permissive (Apache-2.0)

Enoki: structured vectorization and differentiation on modern processor architectures

C++ 1172Updated: 12 mo ago License: Proprietary (Proprietary)

SQL-based streaming analytics platform at scale

Java 1147Updated: 4 y ago License: Permissive (Apache-2.0)

50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak

Shell 1147Updated: 11 mo ago License: Permissive (MIT)

StreamSets Data Collector - Continuous big data and cloud platform ingest infrastructure

Java 1145Updated: 3 y ago License: Permissive (Apache-2.0)

PySpark + Scikit-learn = Sparkit-learn

Python 1135Updated: 1 y ago License: Permissive (Apache-2.0)

Twitter's collection of LZO and Protocol Buffer-related Hadoop, Pig, Hive, and HBase code.

Java 1132Updated: 1 y ago License: Permissive (Apache-2.0)

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

Java 1129Updated: 11 mo ago License: Permissive (Apache-2.0)

A simple and fast linear algebra library for games and graphics

Rust 1116Updated: 11 mo ago License: Permissive (Apache-2.0)