Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

alfhv avatar image
alfhv asked Erick Ramirez answered

Connecting to DSE Spark from java app, getting "Failed to fetch dynamic configuration from DSE"

I have a java code running outside DSE and I want to execute some spark code on DSE cluster (cassandra with analytics spark activated)

code is like this:

SparkConf conf = new SparkConf()
.set("spark.cassandra.connection.host", "my-cassandra-host")
.set("spark.cassandra.auth.username", "my-cassandra-username")
.set("spark.cassandra.auth.password", "my-cassandra-password");

SparConf sparkconf = DseConfiguration.enableDseSupport(conf)
.setMaster("dse://ip-of-master?....");

SparkSession spark = SparkSession.builder().config(sparkconf).appName("my-app").getOrCreate();

I have exported class path from cluster with "dse spark-classpath" all jars are copied locally and added to app classpath.

the master dse:// url is comming from command "dse client-tool spark master-address"

nodetool version:
DSE version: 6.7.14
ReleaseVersion: 4.0.0.6714

When I run code I receive error:

com.datastax.driver.core.Cluster         : Cannot connect with protocol version V5, trying with V4
.m.m.a.ExceptionHandlerExceptionResolver : Resolved [java.io.IOException: Failed to fetch dynamic configuration from DSE]

what means this error related to dynamic configuraiton ?? some port to open or config to add on cluster side ?

dsespark
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

The "dynamic configuration" is a result of calling DseConfiguration.enableDseSupport() in your code.

That method is responsible for connecting to a Cassandra node to retrieve things such as the Spark master address and the digest authentication token used internally in DSE for security.

This is really a Support issue and isn't something I can assist you in a Q&A forum. Please log a ticket with DataStax Support so one of our engineers can work with you directly. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.