incubator-livy | Apache Livy is an open source REST interface
kandi X-RAY | incubator-livy Summary
kandi X-RAY | incubator-livy Summary
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN. Pull requests are welcomed! But before you begin, please check out the Contributing section on the Community page of our website.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of incubator-livy
incubator-livy Key Features
incubator-livy Examples and Code Snippets
Community Discussions
Trending Discussions on incubator-livy
QUESTION
I'm using Spark 3.1.1 which uses Scala 2.12, and the pre-built Livy downloaded from here uses Scala 2.11 (one could find the folder named repl_2.11-jars/
after unzip).
Referred to the comment made by Aliaksandr Sasnouskikh, Livy needs to be rebuilt or it'll throw error {'msg': 'requirement failed: Cannot find Livy REPL jars.'}
even in POST Session.
In the README.md, it mentioned:
By default Livy is built against Apache Spark 2.4.5
If I'd like to rebuild Livy, how could I change the spark version that it is built with?
Thanks in advance.
...ANSWER
Answered 2021-Apr-29 at 19:34You can rebuild Livy passing spark-3.0 profile in maven to create a custom build for spark 3, for example:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install incubator-livy
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page