Kylin | code base is retained for historical interest

by KylinOLAP Java Version: v0.6.5 License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(10)Vulnerabilities Install Support

kandi X-RAY | Kylin Summary

Kylin is a Java library typically used in Big Data, Spark, Hadoop applications. Kylin has build file available, it has a Permissive License and it has low support. However Kylin has 728 bugs and it has 38 vulnerabilities. You can download it from GitHub.

Apache Kylin is an open source Distributed Analytics Engine to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets. Initial contributed by eBay Inc. This code base is retained for historical interest only, please visit Apache Incubator Repo for latest code:

Support

Quality

Security

License

Reuse

Support

Kylin has a low active ecosystem.

It has 561 star(s) with 230 fork(s). There are 96 watchers for this library.

It had no major release in the last 12 months.

There are 11 open issues and 219 have been closed. On average issues are closed in 60 days. There are 10 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of Kylin is v0.6.5

Quality

Kylin has 728 bugs (14 blocker, 5 critical, 332 major, 377 minor) and 3322 code smells.

Security

Kylin has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

Kylin code analysis shows 38 unresolved vulnerabilities (30 blocker, 6 critical, 2 major, 0 minor).

There are 41 security hotspots that need review.

License

Kylin is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

Kylin releases are available to install and integrate.

Build file is available. You can build the component from source.

Kylin saves you 117515 person hours of effort in developing the same functionality from scratch.

It has 124723 lines of code, 4954 functions and 2470 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed Kylin and discovered the below as its top functions. This is intended to give you an instant insight into Kylin implemented functionality, and help decide if they suit your requirements.

Executes the MR job
Check status of yarn job
Get HTTP response
Save a resource
Convert a time interval time to string representation
Power two long
The main entry point to the cube
Sets the reduce task number
Creates a remote scp file
Read ack from the input stream
The main method
Entry point for the job
Merge row key and value
Starts the cube
Determines the output
The start job
Get rows
Update a cube list
Implements the rewrite implementation
Returns all the dimensions of the given SQLDigestDigest
Get the metadata for a project
Main entry point
Demonstrates how to run htable
Parses the cube descriptor
Starts the job
The main entry point

Get all kandi verified functions for this library.

Kylin Key Features

No Key Features are available at this moment for Kylin.

Kylin Examples and Code Snippets

No Code Snippets are available at this moment for Kylin.

Community Discussions

Trending Discussions on Kylin

Update to mapred-default.xml not visible in web UI configuration

Copy particular XML document element to Another XML document using shell commands

sed command to find and replace string with special characters to a string in command line

java.lang.NoSuchMethodError: org.apache.hive.common.util.ShutdownHookManager.addShutdownHook

superset cannot query the kylin's data?

IBM Cloud: How to open Analytics Engine port 7070?

Unable to create a grouping in Apache Kylin

How to create table in partition data in hive?

Install Apache Kylin in custom environment

Unable to start Apache Kylin in HDP 3.0.1

QUESTION

Update to mapred-default.xml not visible in web UI configuration

Asked 2021-Jun-12 at 07:08

I have an Apache Kylin container running in docker. I was getting a Java heap space error in map reduce phase so I tried updating some parameters in Hadoop mapred-default.xml file. After making the changes, I restarted the container but, when I go to Yarn ResourceManager Web UI and then to Configuration:

An xml file is opened, looking like this:

However my new values for the properties that I set inside the mapred-default.xml are not here, it is showing the old values for those properties... Does anyone have any idea why that is happening and what I should do to make it register the new values? I tried restarting the container, but it didn't help...

...

ANSWER

Answered 2021-Jun-12 at 07:08

To override a default value for a property, specify the new value within the tags, inside mapred-site.xml not mapred-default.xml, using the following format:

Source https://stackoverflow.com/questions/67935665

QUESTION

Copy particular XML document element to Another XML document using shell commands

Asked 2020-May-11 at 19:16

I am working on installing Kylin on AWS EMR via shell script . I have an xml file with below content from which I need to copy particular document element to another xml file . This is the manual step I need to automate using shell commands while running the installation shell script.

/etc/hbase/conf/hbase-site.xml

...

ANSWER

Answered 2020-May-11 at 19:16

It is rarely a good idea to try to query XML files with RegEx'es.
Always prefer to use an XML parser!

So you can achieve your given task with xmlstarlet. It is a single program that can get you the data you want in one command from your input ("input.xml"):

Source https://stackoverflow.com/questions/61667880

QUESTION

sed command to find and replace string with special characters to a string in command line

Asked 2020-May-11 at 17:13

I am trying to use sed to find/replace a string with special characters in a .property file.

This is my original line in the file:

...

ANSWER

Answered 2020-May-11 at 17:13

Here:

Source https://stackoverflow.com/questions/61719693

QUESTION

java.lang.NoSuchMethodError: org.apache.hive.common.util.ShutdownHookManager.addShutdownHook

Asked 2019-Dec-18 at 18:42

I'm trying to build a cube on Kylin with Spark as engine type. The cluster contains the following tools:

OS image: 1.0-debian9

Apache Spark 2.4.4 (changed from 1.6.2)

Apache Hadoop 2.7.4

Apache Hive 1.2.1

I'm getting this error while building a cube:

...

ANSWER

Answered 2019-Dec-17 at 19:41

This is not supported use case for Dataproc, if you need to use Spark 2.4.4, then you should use Dataproc 1.4 or 1.5 instead of Dataproc 1.0 that comes with Spark 1.6.2.

Aside this, ShutdownHookManager.addShutdownHook(Ljava/lang/Runnable;)V method was added in Hive 2.3.0, but Spark uses fork of the Hive 1.2.1, that's why you need to use Kylin version that supports Hive 1.2.1.

Regarding duplicate jars, version less hive-common.jar is not a duplicate, it's a symbolic link to the versioned hive-common-1.2.1.jar. You can verify this by listing it:

Source https://stackoverflow.com/questions/59379319

QUESTION

superset cannot query the kylin's data?

Asked 2019-Dec-05 at 08:11

I link to kylin datasource and kylin's data can preview in sqllab,when enter the sql in sqllab,click the run query button,query status always pending and http reponse like

...

ANSWER

Answered 2019-Dec-05 at 08:11

Resovled,the datasource choosed asynchronous query.

Source https://stackoverflow.com/questions/59098621

QUESTION

IBM Cloud: How to open Analytics Engine port 7070?

Asked 2019-Nov-27 at 09:23

I want to use big data services on the IBM Cloud, so I found Analytics Engine(AE) and BigInsight, but unfortunately, BigInsight is going to stop, so I can only choose AE. However, IBM AE is different AWS and GCP big data services, AE prohibits users from having root permissions, so I cannot change some configurations on the clusters, but I want to install Kylin on the cluster, and I need to open Kylin's 7070 port, later, I found that Knox can map ports, but it looks like IBM change it. So how do I open port 7070 for external access? Can I get root permissions? Or is there any other big data service besides "Analytic Engine" and "BigInsight"?

...

ANSWER

Answered 2019-Nov-27 at 09:23

The Analytics Engine service on IBM Cloud can be customized and additional services and packages installed. However, there are restrictions. As you pointed out, it includes that you cannot have root permissions and open ports for security reasons.

To be entirely flexible and have root permissions you could deploy a Hadoop cluster to a Kubernetes or OpenShift cluster or run it on virtual machines.

Source https://stackoverflow.com/questions/59064853

QUESTION

Unable to create a grouping in Apache Kylin

Asked 2019-Nov-14 at 22:39

Hello I am running Apache Kylin apachekylin/apache-kylin-standalone:3.0.0-alpha2 Docker image

I started out by creating two Hive tables one to record store sales and one consisting of store metadata

...

ANSWER

Answered 2019-Nov-14 at 22:39

It turns out the issues is that by default all dimensions in a cube are considered to be derived. That is derived is is selected when I select a measure. It looks like a grouping cannot be created with derived dimensions.

Source https://stackoverflow.com/questions/58649747

QUESTION

How to create table in partition data in hive?

Asked 2019-Nov-11 at 16:50

drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/_impala_insert_staging
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI
[mgupta@sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI
Found 27 items
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201602
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201603
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201604
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201605
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201606
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201607
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201608
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201609
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201610
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201611
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201612
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201701
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201702
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201703
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201704
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201705
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201706
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201707
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201708
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201709
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201710
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201711
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201712
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201801
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201802
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201803
[mgupta@sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601
Found 3 items
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=38527
drwxr-xr-x   - mgupta supergroup          0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=__HIVE_DEFAULT_PARTITION__
[mgupta@sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0
Found 1 items
-rw-r--r--   3 mgupta supergroup    2069014 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0/f9466a0068b906cf-6ace7f8500000049_294515768_data.0.parq
[mgupta@sjc-dev-binn01 ~]$

...

ANSWER

Answered 2019-Nov-11 at 16:50

You may try the steps given below.

Approach 1

Identify the schema (column names and types, including the partitioned column)
Create a hive partitioned table (Make sure to add partition column & delimiter information)
Load data into the partitioned table.(In this case, the loading file will not have partition column as you will hard-code it via the load command)

Source https://stackoverflow.com/questions/49555189

QUESTION

Install Apache Kylin in custom environment

Asked 2019-Sep-27 at 15:02

I'm trying to install Apache Kylin in Ubuntu 16.04.

I installed:

hadoop 3.1.2 in pseudo distributed mode (fs.default.name: hdfs://localhost:9000)
apache hive 3.1.2 and db derby 10.14.2.0 (config hive use db derby)
hbase 1.4.10 in pseudo distributed mode (using hdfs://localhost:9000/hbase)

but when i call:

hbase shell
hbase(main):001:0> list

get error:

...

ANSWER

Answered 2019-Sep-27 at 06:01

Hadoop has a long history and it is complex, so we recommend you to use some well-tested Hadoop Distribution such as CDH and HDP, but not a custom Hadoop environment. If you are do a PoC and want to learn Kylin quickly, please use Docker image https://hub.docker.com/r/apachekylin/apache-kylin-standalone. If want to use Kylin in a more formal Hadoop environment, could you please use a CDH 5.x or HDP 2.x Hadoop Distribution? If you have more question, please contact Kylin community by user mailing list.

Source https://stackoverflow.com/questions/58126981

QUESTION

Unable to start Apache Kylin in HDP 3.0.1

Asked 2019-Sep-23 at 18:59

I'm trying to run Apache Kylin on a cluster with HDP 3.0.1. When I run ./kylin.sh start I get the following output:

(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Retrieving hive dependency... Retrieving hbase dependency... Retrieving hadoop conf dir... Retrieving kafka dependency... Retrieving Spark dependency... Start to check whether we need to migrate acl tables Retrieving hive dependency... Retrieving hbase dependency... Retrieving hadoop conf dir... Retrieving kafka dependency... Retrieving Spark dependency.. . . . . . A new Kylin instance is started by hduser. To stop it, run 'kylin.sh stop' Check the log at /usr/local/apache-kylin-2.6.2-bin-hadoop3/logs/kylin.log Web UI is at http://iservices:7070/kylin

So it seems that Kylin starts correctly.

Then, when I try to access the web UI I get not found error and, in the log file I get this errors:

...

ANSWER

Answered 2019-May-26 at 13:26

You can set $KYLIN_HOME/spark as the SPARK_HOME, if there is no directory named spark, you need to run $KYLIN_HOME/bin/download-spark.sh to download Spark.

Source https://stackoverflow.com/questions/56254032

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install Kylin

You can download it from GitHub.
You can use Kylin like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the Kylin component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: