Big Data Libraries - Page 11

Build, run and manage your data pipelines with Python or SQL on any cloud

Python 340Updated: 2 y ago License: Permissive (Apache-2.0)

Cassandra DB native client written in Rust language. Find 1.x versions on https://github.com/AlexPikalov/cdrs/tree/v.1.x Looking for an async version? - Check WIP https://github.com/AlexPikalov/cdrs-async

Rust 338Updated: 2 y ago License: Permissive (Apache-2.0)

Gathers scalable Tensorflow and Python infrastructure deployment, Husein Go-To for development, 100% Docker.

Jupyter Notebook 337Updated: 3 y ago License: Permissive (MIT)

Sparkling Pandas

Python 337Updated: 4 y ago License: Permissive (Apache-2.0)

Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows locally or on a cluster.

Java 337Updated: 2 y ago License: Proprietary (Proprietary)

HDFS-alike in Go. Written in 2014 to learn the language and get a job.

Go 336Updated: 4 y ago License: No License (No License)

Django Cassandra Engine - the Cassandra backend for Django

Python 335Updated: 2 y ago License: Permissive (BSD-2-Clause)

Next-generation web analytics processing with Scala, Spark, and Parquet.

JavaScript 335Updated: 4 y ago License: Permissive (Apache-2.0)

Scala 333Updated: 4 y ago License: No License (No License)

Harmonious distributed data analysis in Rust.

Rust 332Updated: 4 y ago License: Permissive (Apache-2.0)

CentOS Cloud Instance SIG: Metadata to build & release instances

Shell 332Updated: 4 y ago License: No License (No License)

A boilerplate for writing PySpark Jobs

Python 331Updated: 4 y ago License: No License (No License)

Create clusters of VMs on the cloud and configure them with Ansible.

Python 329Updated: 2 y ago License: Strong Copyleft (GPL-3.0)

Examples for learning spark

Java 328Updated: 4 y ago License: Permissive (MIT)

<<THIS REPOSITORY IS DEPRECATED>> The HTTP Archive provides information about website performance such as # of HTTP requests, use of gzip, and amount of JavaScript. This information is recorded over time revealing trends in how the Internet is performing. Built using Open Source software, the code and data are available to everyone allowing researchers large and small to work from a common base.

PHP 328Updated: 3 y ago License: Proprietary (Proprietary)

xichuan的学习总结笔记,覆盖了java、spring、java其他常用框架,以及大数据相关组件等📚

Java 328Updated: 2 y ago License: No License (No License)

[CVPR 2022 Oral] QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection

Python 327Updated: 2 y ago License: Permissive (MIT)

The Kinesis Scaling Utility is designed to give you the ability to scale Amazon Kinesis Streams in the same way that you scale EC2 Auto Scaling groups – up or down by a count or as a percentage of the total fleet. You can also simply scale to an exact number of Shards. There is no requirement for you to manage the allocation of the keyspace to Shards when using this API, as it is done automatically.

Java 325Updated: 3 y ago License: Permissive (Apache-2.0)

Fast integer compression in C using the StreamVByte codec

C 324Updated: 2 y ago License: Permissive (Apache-2.0)

Native, optimized access to HBase Data through Spark SQL/Dataframe Interfaces

Scala 320Updated: 4 y ago License: Permissive (Apache-2.0)

A tool for monitoring and tuning Spark jobs for efficiency.

Scala 319Updated: 4 y ago License: Permissive (Apache-2.0)

Reactive Hive Toolkit for Streaming Platforms

Kotlin 318Updated: 4 y ago License: Permissive (Apache-2.0)

A Bulk Data Pipeline out of Cassandra

Java 316Updated: 4 y ago License: Permissive (Apache-2.0)

Mirror of Apache HttpCore

Java 315Updated: 2 y ago License: Permissive (Apache-2.0)

Train and run Pytorch models on Apache Spark.

Python 313Updated: 2 y ago License: Permissive (MIT)

Apache hadoop management system

PHP 313Updated: 2 y ago License: Strong Copyleft (GPL-3.0)

Fast. Scalable. Powerful. The Blockchain for Web3

C++ 313Updated: 2 y ago License: Proprietary (Proprietary)

Crunch is an Apache TLP now, and lives at http://crunch.apache.org/

Java 312Updated: 4 y ago License: Permissive (Apache-2.0)

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

Scala 312Updated: 3 y ago License: Permissive (Apache-2.0)

Powered by Spark Streaming & Siddhi

Java 311Updated: 4 y ago License: Permissive (Apache-2.0)

PHP 311Updated: 4 y ago License: Permissive (MIT)

Distributed SQL Engine in Python using Dask

Python 311Updated: 2 y ago License: Permissive (MIT)

Serverless proxy for Spark cluster

Scala 310Updated: 4 y ago License: Permissive (Apache-2.0)

Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.

Scala 307Updated: 4 y ago License: Permissive (Apache-2.0)

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

Java 306Updated: 3 y ago License: Permissive (Apache-2.0)

Spark library for easy MongoDB access

Scala 306Updated: 4 y ago License: Permissive (Apache-2.0)

Drop-in replacement of LINQ aggregation operations extremely faster with SIMD.

C# 306Updated: 2 y ago License: Permissive (MIT)

TensorFlow on Spark

Python 305Updated: 4 y ago License: No License (No License)

std::experimental::simd for GCC [ISO/IEC TS 19570:2018]

C++ 304Updated: 4 y ago License: Proprietary (Proprietary)

Go 304Updated: 4 y ago License: Permissive (Apache-2.0)

Open single and half precision gemm implementations

C 303Updated: 4 y ago License: Permissive (MIT)

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks

Python 302Updated: 2 y ago License: Permissive (Apache-2.0)

my build machine for bromite development

Shell 302Updated: 2 y ago License: Strong Copyleft (GPL-3.0)

大数据采集,抽取平台

Java 301Updated: 2 y ago License: Permissive (Apache-2.0)

Easy to use library to bring Tensorflow on Apache Spark

Python 299Updated: 2 y ago License: Permissive (MIT)

Connect Spark to HBase for reading and writing data with ease

Scala 299Updated: 2 y ago License: Permissive (Apache-2.0)

An open platform and marketplace for distributed computations

Rust 299Updated: 2 y ago License: Strong Copyleft (GPL-3.0)

Standalone VM using LLVM JIT

C++ 299Updated: 4 y ago License: Proprietary (Proprietary)

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Java 298Updated: 2 y ago License: Permissive (Apache-2.0)

Demonstrates a software (CPU) based approach to occllusion culling using multi-threading and SIMD instructions to improve performance.

C++ 298Updated: 4 y ago License: Permissive (Apache-2.0)