DataStax Academy FAQ

DataStax Academy migrated to a new learning management system (LMS) in July 2020. We are also moving to a new Cassandra Certification process so there are changes to exam bookings, voucher system and issuing of certificates.

Check out the Academy FAQ pages for answers to your questions:


question

james.kavanaghrsa_179397 avatar image
james.kavanaghrsa_179397 asked ·

Obtaining a CassandraConnector from cluster config stored in SparkSession?

Hi there,


I have a spark session that I use to connect to two different clusters by using:

spark.setCassandraConf("myremotecluster", myRemoteClusterConf)

Then later I do the following:

spark.read.cassandraFormat(cluster = "myremotecluster",
                           keyspace = "myremotekeyspace",
                           table = "myremotetable").load

What I would like to do is use this configuration to do the following:

// get the cluster config from the spark session somehow here //
// then use it
CassandraConnector(myRemoteClusterConf).withSessionDo { s => .... }

Is there an easy way to read this configuration back from the spark session? I can't seem to find one :-/

I can cache it somewhere myself but would be great to be able to grab it from the spark if possible.


Thanks,

James



spark-connector
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

0 Answers