Hadoop Libraries - Page 5

Large-scale event processing with Akka Persistence and Apache Spark

Scala 277Updated: 2 y ago License: Permissive (Apache-2.0)

ETL Library for Machine Learning - data pipelines, data munging and wrangling

Java 275Updated: 4 y ago License: Permissive (Apache-2.0)

High Throughput Real-time Stream Processing Framework

C++ 275Updated: 4 y ago License: Permissive (Apache-2.0)

Hops Hadoop is a distribution of Apache Hadoop with distributed metadata.

Java 273Updated: 2 y ago License: Permissive (Apache-2.0)

Fundamentals of Spark with Python (using PySpark), code examples

Jupyter Notebook 273Updated: 2 y ago License: Permissive (MIT)

Complex Event Processing on top of Kafka Streams

Java 267Updated: 4 y ago License: Permissive (Apache-2.0)

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Java 267Updated: 2 y ago License: Permissive (Apache-2.0)

Companion repository for Web Development With Node and Express, 2nd Edition (O'Reilly).

JavaScript 263Updated: 2 y ago License: No License (No License)

Ferry lets you define, run, and deploy big data applications on AWS, OpenStack, and your local machine using Docker

Python 256Updated: 4 y ago License: Permissive (Apache-2.0)

Slimming down jars since 2016

Java 256Updated: 4 y ago License: Permissive (Apache-2.0)

An efficient updatable key-value store for Apache Spark

Scala 252Updated: 4 y ago License: Permissive (Apache-2.0)

Apache Cassandra cluster orchestration tool for the command line

Python 249Updated: 3 y ago License: Permissive (Apache-2.0)

Flume Source to import data from SQL Databases

Java 248Updated: 4 y ago License: Permissive (Apache-2.0)

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

Go 248Updated: 2 y ago License: Permissive (Apache-2.0)

如果你在从事大数据BI的工作，想对比一下MySQL、GreenPlum、Elasticsearch、Hive、Spark SQL、Presto、Impala、Drill、HAWQ、Druid、Pinot、Kylin、ClickHouse、Kudu等不同实现方案之间的表现，那你就需要一份标准的数据进行测试，这个开源项目就是为了生成这样的标准数据。

Java 247Updated: 4 y ago License: Permissive (Apache-2.0)

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Go 246Updated: 4 y ago License: Permissive (MIT)

AntDB is a distributed database to provide both write-scalability and massively parallel processing. The mirror git repository is at https://gitee.com/adbsql/antdb.

C 245Updated: 2 y ago License: Permissive (Apache-2.0)

Fast and efficient batch computation engine for complex analysis and reporting of massive datasets on Hadoop

Java 243Updated: 5 y ago License: Permissive (Apache-2.0)

基于Docker构建的Hadoop开发测试环境，包含Hadoop，Hive，HBase，Spark

Shell 243Updated: 4 y ago License: No License (No License)

Apache Trafodion

C++ 243Updated: 4 y ago License: No License (No License)

A C++ implementaton of MapReduce without distributed filesystem

C++ 240Updated: 4 y ago License: Proprietary (Proprietary)

Hadoop (Utilities, Patches and Examples)

Python 238Updated: 2 y ago License: No License (No License)

BigDataPlatform：基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。

Java 238Updated: 5 y ago License: Strong Copyleft (GPL-3.0)

A toolkit providing a uniform interface for connecting to and extracting data from a wide variety of (potentially remote) data stores (including HDFS, Hive, Presto, MySQL, etc).

Python 238Updated: 2 y ago License: Permissive (MIT)

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Scala 237Updated: 2 y ago License: Permissive (MIT)

Tool for Java codebases that will help you identify the God Classes you should refactor first.

Java 237Updated: 2 y ago License: Permissive (Apache-2.0)

Facebook's Hive UDFs

Java 235Updated: 3 y ago License: Permissive (Apache-2.0)

推荐项目（实时推荐和离线推荐）

Java 235Updated: 2 y ago License: No License (No License)

Cloud9 is a Hadoop toolkit for working with big data

Java 234Updated: 4 y ago License: No License (No License)

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Scala 233Updated: 2 y ago License: Permissive (Apache-2.0)

大数据相关框架实战项目(Hadoop, Spark, Storm, Flink)

Java 232Updated: 2 y ago License: No License (No License)

An end-to-end machine learning and data mining framework on Hadoop

Java 232Updated: 2 y ago License: Permissive (Apache-2.0)

一个开源、成体系的大数据学习教程。spark学习 hadoop hive hbase flink教程 linux 从入门到精通

Shell 232Updated: 2 y ago License: No License (No License)

LascoDan(Korea Scala Group) scala study

Scala 232Updated: 4 y ago License: No License (No License)

A collection of pentest tools and resources targeting Hadoop environments

Python 228Updated: 4 y ago License: No License (No License)

J2EE学习以及Linux组件学习的日常总结，适合想了解和温习基础知识的童鞋。目前计划包含的内容有设计模式、Springboot、SpringCloud；以及Linux开源组件Redis、Kafka、Nginx、ElasticSearch、Hadoop、Zookeeper等

JavaScript 227Updated: 4 y ago License: Permissive (MIT)

Hive JDBC "uber" or "standalone" jar based on the latest Apache Hive version

Java 226Updated: 3 y ago License: No License (No License)

Apache Spark™ and Scala Workshops

HTML 225Updated: 4 y ago License: Permissive (Apache-2.0)

Write Hadoop jobs in JRuby

Ruby 224Updated: 4 y ago License: No License (No License)

Visualize your HDFS cluster usage

JavaScript 224Updated: 4 y ago License: Permissive (Apache-2.0)

Automated turndown of Kubernetes clusters on specific schedules.

Go 224Updated: 2 y ago License: Permissive (Apache-2.0)

Dockerfiles for building a storm cluster.

Shell 223Updated: 4 y ago License: Permissive (Apache-2.0)

GraphChi's Java version

Java 219Updated: 4 y ago License: No License (No License)

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Java 218Updated: 4 y ago License: Permissive (Apache-2.0)

DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server

Java 217Updated: 4 y ago License: Permissive (Apache-2.0)

presto hbase connector 组件基于Presto Connector接口规范实现，用来给Presto增加查询HBase的功能。相比其他开源版本的HBase Connector，我们的性能要快10到100倍以上。

Java 216Updated: 4 y ago License: Permissive (Apache-2.0)

A Python MapReduce and HDFS API for Hadoop

Python 216Updated: 4 y ago License: Permissive (Apache-2.0)

How to do Unsupervised Clustering with Keras

Jupyter Notebook 214Updated: 4 y ago License: Proprietary (Proprietary)

A set of scripts and config files to run a Cassandra cluster from Docker

Shell 212Updated: 4 y ago License: Permissive (Apache-2.0)

Spark、Flink等离线任务的调度以及实时任务的监控

Java 210Updated: 4 y ago License: Permissive (Apache-2.0)