Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

randas avatar image
randas asked Erick Ramirez answered

Can't see any keyspaces with spark.read.format [duplicate of #11034]

I'm using spark 3.0.2, cassandra connector 3.0.0, cassandra 3.11.2

I can retrieve from cassandra using :

val res = CassandraConnector(conf).withSessionDo { session =>
session.execute("select * From test.words") }

but through :

val typeData = spark.read.format("org.apache.spark.sql.cassandra")
  .options(Map( "table" -> "words", "keyspace" -> "test"))
  .load()

got can't see test keyspaces.
I'm deploying my application as a fat jar "which is working well locally in both way" but on deployment environment is not working using the spark.read.format.

is there any connection I could check for the spark shell to see cassandra through this way : spark.read.format ? or there's onother way I could retrieve from the CassandraConnector as a DF -not result set-

spark-cassandra-connector
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

[Duplicate of question #11034]

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.