spark-cassandra-connector | DataStax Spark Cassandra Connector

Is Datastax Cassandra community edition integration with Spark community edition using spark-cassandra-connector community edition node aware or is this feature reserved for Enterprise editions only?

By node awareness I mean if Spark will send job execution to the nodes owning the data

I am very new to apache spark and I just have to fetch a table from cassandra database, Below I have appended the data to debug the situation, Please help and thanks in advance. Cassandra Node:192.168.56.10 Spark Node: 192.168.56.10

Cassandra Table to be fetched: dev.device {keyspace.table_name}

Access pyspark with connection to cassandra:

I am a complete beginner to all this stuff in general so pardon if I'm missing some totally obvious step. I installed spark 3.1.2 and cassandra 3.11.11 and I'm trying to connect both of them through this guide I found where I made a fat jar for execution. In the link I posted when they execute the spark-shell command with the jar file, there's a line which occurs at the start.

INFO SparkContext: Added JAR file:/home/chbatey/dev/tmp/spark-cassandra-connector/spark-cassandra-connector-java/target/scala-2.10/spark-cassandra-connector-java-assembly-1.2.0-SNAPSHOT.jar at http://192.168.0.34:51235/jars/spark-15/01/26 16:16:10 INFO SparkILoop: Created spark context..

I followed all of the steps properly but it doesn't show any line like that in my shell. To confirm that it hasn't been added I try the sample program on that website and it throws an error

java.lang.NoClassDefFoundError: com/datastax/spark/connector/util/Logging

What should I do? I'm using spark-cassandra-connector-3.1.0

Although the latest spark-cassandra-connector from DataStax states it supports reading/writing TTL and WRITETIME I am still receiving a SQL undefined function error.

Using Databricks with library com.datastax.spark:spark-cassandra-connector-assembly_2.12:3.1.0 and a Spark Config for CassandraSparkExtensions on a 9.1 LTS ML (includes Apache Spark 3.1.2, Scala 2.12) Cluster. CQL version 3.4.5.

Small question regarding IntelliJ and Maven pom.xml please.

In several online tutorials, I saw when the user is using IntelliJ, Maven, and has a dependency which version is outdated, got a nice warning.

For instance, in this piece of pom:

I am trying to get data using dataframes from cassandra by using spark-cassandra-connector but getting below exception.

Note: Connection is successful to cassandra.
Spark version: 2.4.1
spark-cassandra-connector version: 2.5.1

I'm new to Cassandra and Pyspark, initially I installed cassandra version 3.11.1, openjdk 1.8, pyspark 3.x and scala 1.12. I was getting a lot of errors as shown below after running my python server.

Create table -

I created Docker containers in which I installed Apache Spark 3.1.2 (Hadoop 3.2) that host a ThriftServer which is configured to access Cassandra via the spark-cassandra-connector(3.1.0). Each of these services is running in it's own container. So I got 5 containers up (1x spark master, 2x spark worker, 1x spark thriftserver, 1x cassandra) which are configured to live in the same network via docker-compose. I use the beeline client from Apache Hive(1.2.1) to query the database. Everything works fine, except for querying a field in Cassandra with the type timestamp.