question

alfhv avatar image
alfhv asked Erick Ramirez answered

Connecting to DSE Spark from java app, getting "Failed to fetch dynamic configuration from DSE"

I have a java code running outside DSE and I want to execute some spark code on DSE cluster (cassandra with analytics spark activated)

code is like this:

SparkConf conf = new SparkConf()
.set("spark.cassandra.connection.host", "my-cassandra-host")
.set("spark.cassandra.auth.username", "my-cassandra-username")
.set("spark.cassandra.auth.password", "my-cassandra-password");

SparConf sparkconf = DseConfiguration.enableDseSupport(conf)
.setMaster("dse://ip-of-master?....");

SparkSession spark = SparkSession.builder().config(sparkconf).appName("my-app").getOrCreate();

I have exported class path from cluster with "dse spark-classpath" all jars are copied locally and added to app classpath.

the master dse:// url is comming from command "dse client-tool spark master-address"

nodetool version:
DSE version: 6.7.14
ReleaseVersion: 4.0.0.6714

When I run code I receive error:

com.datastax.driver.core.Cluster         : Cannot connect with protocol version V5, trying with V4
.m.m.a.ExceptionHandlerExceptionResolver : Resolved [java.io.IOException: Failed to fetch dynamic configuration from DSE]

what means this error related to dynamic configuraiton ?? some port to open or config to add on cluster side ?

dsespark
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

The "dynamic configuration" is a result of calling DseConfiguration.enableDseSupport() in your code.

That method is responsible for connecting to a Cassandra node to retrieve things such as the Spark master address and the digest authentication token used internally in DSE for security.

This is really a Support issue and isn't something I can assist you in a Q&A forum. Please log a ticket with DataStax Support so one of our engineers can work with you directly. Cheers!

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.