Building a Stream Processing Application using open source libraries

Build Stream Processing

Big Data 

New List

Apache Spark - A unified analytics engine for large-scale data processing

Apache License 2.0

spark

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library,  for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

xgboost

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Other

data-science-ipython-notebooks

simdjson

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in. 

luigi

The official home of the Presto distributed SQL query engine for big data

presto

PredictionIO, a machine learning server for developers and ML engineers.

attic-predictionio

hadoop

flink learning blog. http://www.54tianzhisheng.cn/  含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

flink-learning

Apache Druid: a high performance real-time analytics database.

All Big Data Solution Kits

Open Weaver – Develop Applications Faster with Open Source

kandi

Community and Support

Company

Follow