I have a java code running outside DSE and I want to execute some spark code on DSE cluster (cassandra with analytics spark activated)
code is like this:
SparkConf conf = new SparkConf() .set("spark.cassandra.connection.host", "my-cassandra-host") .set("spark.cassandra.auth.username", "my-cassandra-username") .set("spark.cassandra.auth.password", "my-cassandra-password"); SparConf sparkconf = DseConfiguration.enableDseSupport(conf) .setMaster("dse://ip-of-master?...."); SparkSession spark = SparkSession.builder().config(sparkconf).appName("my-app").getOrCreate();
I have exported class path from cluster with "dse spark-classpath" all jars are copied locally and added to app classpath.
the master dse:// url is comming from command "dse client-tool spark master-address"
DSE version: 6.7.14
When I run code I receive error:
com.datastax.driver.core.Cluster : Cannot connect with protocol version V5, trying with V4 .m.m.a.ExceptionHandlerExceptionResolver : Resolved [java.io.IOException: Failed to fetch dynamic configuration from DSE]
what means this error related to dynamic configuraiton ?? some port to open or config to add on cluster side ?