I'm using spark 3.0.2, cassandra connector 3.0.0, cassandra 3.11.2
I can retrieve from cassandra using :
val res = CassandraConnector(conf).withSessionDo { session =>
session.execute("select * From test.words") }
but through :
val typeData = spark.read.format("org.apache.spark.sql.cassandra")
.options(Map( "table" -> "words", "keyspace" -> "test"))
.load()
got can't see test keyspaces.
I'm deploying my application as a fat jar "which is working well locally in both way" but on deployment environment is not working using the spark.read.format.
is there any connection I could check for the spark shell to see cassandra through this way : spark.read.format ? or there's onother way I could retrieve from the CassandraConnector as a DF -not result set-
