Big Data Libraries - Page 7

This page is a summary to keep the track of Hadoop related projects, and relevant projects around Big Data scene focused on the open source, free software environment.

HTML 667Updated: 4 y ago License: Permissive (Apache-2.0)

A wide linear algebra crate for games and graphics.

Rust 665Updated: 2 y ago License: No License (No License)

Fast, accurate and scalable probabilistic data linkage using your choice of SQL backend

Python 664Updated: 2 y ago License: Permissive (MIT)

OpenDAL: Access data freely, painlessly, and efficiently

Rust 664Updated: 2 y ago License: Permissive (Apache-2.0)

Apache Flink Training Excercises

Java 663Updated: 2 y ago License: Permissive (Apache-2.0)

(Finished) Geek Time Data Analysis Practical 45 Lecture - Detailed notes containing markdown images mind map code data can be read directly code test

Python 663Updated: 2 y ago License: Proprietary (Proprietary)

P(R*_{3, 0, 1}) specialized SIMD Geometric Algebra Library

C++ 660Updated: 2 y ago License: Permissive (MIT)

Scalable, redundant, and distributed object store for Apache Hadoop

Java 658Updated: 2 y ago License: Permissive (Apache-2.0)

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Python 657Updated: 2 y ago License: Permissive (MIT)

BlinkDB: Sub-Second Approximate Queries on Very Large Data.

Scala 648Updated: 2 y ago License: Permissive (Apache-2.0)

Nessie: Transactional Catalog for Data Lakes with Git-like semantics

Java 644Updated: 2 y ago License: Permissive (Apache-2.0)

R frontend for Spark

R 643Updated: 4 y ago License: Permissive (Apache-2.0)

TonY is a framework to natively run deep learning frameworks on Apache Hadoop.

Java 640Updated: 4 y ago License: Proprietary (Proprietary)

pyspark🍒🥭 is delicious，just eat it!😋😋

Python 636Updated: 2 y ago License: No License (No License)

Spark reference applications

Scala 633Updated: 4 y ago License: Proprietary (Proprietary)

Quantcast File System

C++ 632Updated: 2 y ago License: Permissive (Apache-2.0)

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

Java 632Updated: 2 y ago License: Permissive (Apache-2.0)

Lazydata: Scalable data dependencies for Python projects

Python 629Updated: 4 y ago License: Permissive (Apache-2.0)

Golang framework for streaming ETL, observability data pipeline, and event processing apps

Go 629Updated: 2 y ago License: Proprietary (Proprietary)

DataFusion has now been donated to the Apache Arrow project

Rust 627Updated: 2 y ago License: Permissive (Apache-2.0)

💎🔥大数据学习笔记

Java 622Updated: 2 y ago License: No License (No License)

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

Python 615Updated: 4 y ago License: Strong Copyleft (GPL-3.0)

Expressive Vector Engine - SIMD in C++ Goes Brrrr

C++ 615Updated: 2 y ago License: Permissive (BSL-1.0)

Spring for Apache Hadoop is a framework for application developers to take advantage of the features of both Hadoop and Spring.

Java 612Updated: 4 y ago License: No License (No License)

Low-code metrics store, modern open-source alternative to Looker

JavaScript 612Updated: 2 y ago License: Proprietary (Proprietary)

Mirror of Apache Giraph

Java 610Updated: 2 y ago License: Permissive (Apache-2.0)

Distributed Neural Networks for Spark

Scala 608Updated: 4 y ago License: Permissive (MIT)

Apache ORC - the smallest, fastest columnar storage for Hadoop workloads

HTML 607Updated: 2 y ago License: Permissive (Apache-2.0)

C# bindings for lemire/simdjson (and full C# port)

C# 605Updated: 2 y ago License: Permissive (Apache-2.0)

Python 605Updated: 2 y ago License: No License (No License)

Web UI for Trino, Hive and SparkSQL

Java 602Updated: 2 y ago License: Permissive (Apache-2.0)

Library for specialized dense and sparse matrix operations, and deep learning primitives.

C 597Updated: 3 y ago License: Permissive (BSD-3-Clause)

A tool to graphically visualize SIMD code

JavaScript 595Updated: 2 y ago License: Permissive (BSD-3-Clause)

Jupyter Notebook & Data Associated with my Tutorial video on the Python NumPy Library

Jupyter Notebook 594Updated: 2 y ago License: No License (No License)

Redshift data source for Apache Spark

Scala 593Updated: 2 y ago License: Permissive (Apache-2.0)

An open protocol for secure data sharing

Scala 592Updated: 2 y ago License: Permissive (Apache-2.0)

A distributed system designed to ingest and process time series data

Java 592Updated: 3 y ago License: Permissive (Apache-2.0)

BigData Project 大数据项目由浅入深

Java 588Updated: 2 y ago License: Strong Copyleft (GPL-3.0)

Hadoop library for large-scale data processing, now an Apache Incubator project

Java 588Updated: 5 y ago License: No License (No License)

Hadoop library for large-scale data processing, now an Apache Incubator project

Java 588Updated: 4 y ago License: No License (No License)

Today I Learned

JavaScript 586Updated: 2 y ago License: No License (No License)

A wide linear algebra crate for games and graphics.

Rust 580Updated: 3 y ago License: No License (No License)

Fast Disk-Based Parallelized Data Manipulation Framework for Larger-than-RAM Data

R 580Updated: 3 y ago License: Proprietary (Proprietary)

A tutorial on the most important features and idioms of Scala that you need to use Spark's Scala APIs.

Jupyter Notebook 579Updated: 3 y ago License: Permissive (Apache-2.0)

A lightweight high performance tensor algebra framework for modern C++

C++ 578Updated: 2 y ago License: Permissive (MIT)

Python bindings for the simdjson project.

Python 577Updated: 2 y ago License: Proprietary (Proprietary)

DataStax C# Driver for Apache Cassandra

C# 576Updated: 2 y ago License: Permissive (Apache-2.0)

Fast Disk-Based Parallelized Data Manipulation Framework for Larger-than-RAM Data

R 574Updated: 3 y ago License: Proprietary (Proprietary)

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

Python 573Updated: 2 y ago License: No License (No License)

A vector with a fixed capacity. (Rust)

Rust 572Updated: 2 y ago License: Permissive (Apache-2.0)