Trying to connect from spark in one data center (only used for processing) to cassandra (data is present) in other data center. Would please advise how this can be achieved. using DSE 5.1.17.On submit spark job it is always connecting to cassandra on spark node.
Note: User used for connecting Spark is different from Cassandra user.
Able connect to C* and Spark if user is same
[UPDATE]
Environment.
- sparkUser - Present in spark cluster
- cassUser - Present in C* and spark cluster
- "superusers" = "Y"
Fails
dse -u sparkUser -p sparkPwd spark --master "dse://sparkIP" --conf spark.cassandra.connection.host=cassIP --conf spark.cassandra.auth.username=cassUser --conf spark.cassandra.auth.password=cassPwd --conf spark.hadoop.cassandra.host=cassIP
Succeeds
dse -u cassUser -p cassPwd spark --master "dse://sparkIP" --conf spark.cassandra.connection.host=cassIP --conf spark.cassandra.auth.username=cassUser --conf spark.cassandra.auth.password=cassPwd --conf spark.hadoop.cassandra.host=cassIP
Expectation
Command to succeed with different user. i.e. option 1