spark-as-service-using-embedded-server | application comes as Spark2.1-as-Service-Provider | REST library
kandi X-RAY | spark-as-service-using-embedded-server Summary
kandi X-RAY | spark-as-service-using-embedded-server Summary
This application comes as Spark2.1-as-Service-Provider using an embedded, Reactive-Streams-based, fully asynchronous HTTP server
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of spark-as-service-using-embedded-server
spark-as-service-using-embedded-server Key Features
spark-as-service-using-embedded-server Examples and Code Snippets
Community Discussions
Trending Discussions on spark-as-service-using-embedded-server
QUESTION
I've followed a blog for exposing Spark as a WebService using Akka HTTP.
The code works fine when I'm running it independently but fails when I tried to connect it to a Spark Standalone cluster. I did some analysis and many suggested to set host entry and spark-env.sh correctly. I have done all those and still, it's not working.
Note: My Spark Version and the Spark version in my pom is also the same and I can submit a job to the same cluster using "spark-submit".
Host File
...ANSWER
Answered 2019-Jul-14 at 15:09I got the above issue fixed. Sharing the root cause for others reference and to save time.
The root cause is, I am using Spark 2.4.3 and it's internally using Scala 2.11 and the Scala project I'm using is having Scala 2.12. This was creating all the problems the whole time.
I have downgraded the Scala version in my project to get it to work.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install spark-as-service-using-embedded-server
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page