Big Data Libraries - Page 3

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

Go 2089Updated: 2 y ago License: Permissive (Apache-2.0)

Apache Parquet

Java 2042Updated: 2 y ago License: Permissive (Apache-2.0)

Embree ray tracing kernels repository.

C++ 2024Updated: 2 y ago License: Permissive (Apache-2.0)

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

Java 2023Updated: 2 y ago License: No License (No License)

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Java 2005Updated: 2 y ago License: Permissive (MIT)

Compile-time Language Integrated Queries for Scala

Scala 1992Updated: 3 y ago License: Permissive (Apache-2.0)

Make stream processing easier! easy-to-use stream processing application development framework and one-stop stream processing operation platform

Java 1971Updated: 2 y ago License: Permissive (Apache-2.0)

Node.js bindings for librdkafka

JavaScript 1969Updated: 2 y ago License: Permissive (MIT)

Deep Learning Pipelines for Apache Spark

Python 1968Updated: 2 y ago License: Permissive (Apache-2.0)

Easy Machine Learning is a general-purpose dataflow-based system for easing the process of applying machine learning algorithms to real world tasks.

Java 1958Updated: 2 y ago License: Permissive (Apache-2.0)

esProc SPL is a scripting language for data processing, with well-designed rich library functions and powerful syntax, which can be executed in a Java program through JDBC interface and computing independently.

Java 1951Updated: 2 y ago License: Permissive (Apache-2.0)

Apache Hadoop docker image

Shell 1940Updated: 2 y ago License: No License (No License)

TensorFlow binaries supporting AVX, FMA, SSE

Shell 1938Updated: 2 y ago License: No License (No License)

Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.

Java 1925Updated: 2 y ago License: Permissive (Apache-2.0)

基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法

Java 1921Updated: 2 y ago License: Permissive (Apache-2.0)

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

C# 1905Updated: 2 y ago License: Permissive (MIT)

:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop

Java 1902Updated: 2 y ago License: Permissive (Apache-2.0)

DataStax Spark Cassandra Connector

Scala 1902Updated: 2 y ago License: Permissive (Apache-2.0)

深圳地铁大数据客流分析系统🚇🚄🌟

Scala 1871Updated: 2 y ago License: Proprietary (Proprietary)

Implementations of SIMD instruction sets for systems which don't natively support them.

C 1827Updated: 2 y ago License: Permissive (MIT)

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

Java 1819Updated: 4 y ago License: Permissive (Apache-2.0)

Production Ready Data Integration Product, documentation：

Java 1819Updated: 3 y ago License: Permissive (Apache-2.0)

Apache Drill is a distributed MPP query layer for self describing data

Java 1801Updated: 2 y ago License: Permissive (Apache-2.0)

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Java 1798Updated: 2 y ago License: Permissive (Apache-2.0)

a JIT assembler for x86(IA-32)/x64(AMD64, x86-64) MMX/SSE/SSE2/SSE3/SSSE3/SSE4/FPU/AVX/AVX2/AVX-512 by C++ header

C++ 1785Updated: 2 y ago License: Permissive (BSD-3-Clause)

jdbi is designed to provide convenient tabular data access in Java; including templated SQL, parameterized and strongly typed queries, and Streams integration

Java 1782Updated: 2 y ago License: Permissive (Apache-2.0)

Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads

Java 1748Updated: 2 y ago License: Permissive (Apache-2.0)

C++ wrappers for SIMD intrinsics and parallelized, optimized mathematical functions (SSE, AVX, AVX512, NEON, SVE))

C++ 1747Updated: 2 y ago License: Permissive (BSD-3-Clause)

Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.

Scala 1731Updated: 2 y ago License: Permissive (Apache-2.0)

📽 Highly Optimized Graphics Math (glm) for C

C 1711Updated: 2 y ago License: Permissive (MIT)

Mirror of Apache Kudu

C++ 1709Updated: 2 y ago License: Permissive (Apache-2.0)

A large-scale entity and relation database supporting aggregation of properties

Java 1700Updated: 2 y ago License: Permissive (Apache-2.0)

ApacheCN 开源组织：公告、介绍、成员、活动、交流方式

CSS 1694Updated: 2 y ago License: Proprietary (Proprietary)

Elassandra = Elasticsearch + Apache Cassandra

Java 1667Updated: 2 y ago License: Permissive (Apache-2.0)

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

Scala 1631Updated: 2 y ago License: Permissive (Apache-2.0)

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Python 1622Updated: 2 y ago License: Permissive (Apache-2.0)

Numpy exercises.

Python 1620Updated: 2 y ago License: Permissive (MIT)

Python interface to Hive and Presto. 🐝

Python 1609Updated: 2 y ago License: Proprietary (Proprietary)

生产环境的海量数据计算产品，文档地址：

Java 1601Updated: 4 y ago License: Permissive (Apache-2.0)

Home of the community managed version of Presto, the distributed SQL query engine for big data, under the auspices of the Presto Software Foundation.

Java 1595Updated: 4 y ago License: Permissive (Apache-2.0)

Distributed Deep learning with Keras & Spark

Python 1560Updated: 2 y ago License: Permissive (MIT)

Apache Atlas

Java 1549Updated: 2 y ago License: Permissive (Apache-2.0)

The Universal Storage Engine

C++ 1546Updated: 2 y ago License: Permissive (MIT)

MongoDB Connector for Hadoop

Java 1521Updated: 2 y ago License: No License (No License)

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Jupyter Notebook 1521Updated: 2 y ago License: Proprietary (Proprietary)

YTsaurus is a scalable and fault-tolerant open-source big data platform.

C++ 1520Updated: 2 y ago License: Proprietary (Proprietary)

A Scala kernel for Jupyter

Scala 1516Updated: 2 y ago License: Permissive (BSD-3-Clause)

SIMD for humans

Rust 1507Updated: 2 y ago License: Weak Copyleft (MPL-2.0)

Open-source graph database, built for real-time streaming data, compatible with Neo4j.

C++ 1494Updated: 2 y ago License: Proprietary (Proprietary)

Code to accompany Advanced Analytics with Spark from O'Reilly Media

Scala 1485Updated: 2 y ago License: Proprietary (Proprietary)