Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

scano_183208 avatar image
scano_183208 asked ·

spark-submit unable to connect to DSEFS

When i try the following command:

dse spark-submit --class main.ReportExporter --conf spark.master=dse:/// dsefs:/sparkbin/report-exporter-assembly-0.2.jar

I keep receiving the following error:

Exception in thread "main" com.datastax.bdp.fs.client.DseFsConnectionException: Could not connect to DSEFS. Last endpoint tried: 192.168.21.4:5598
    at com.datastax.bdp.fs.client.RestClientProxy.<init>(RestClientProxy.scala:161)
    at com.datastax.bdp.fs.client.DseFsClient.liftedTree1$1(DseFsClient.scala:62)
    at com.datastax.bdp.fs.client.DseFsClient.<init>(DseFsClient.scala:62)
    at com.datastax.bdp.fs.hadoop.DseFileSystem.initialize(DseFileSystem.scala:129)

I am having trouble finding to root cause of this and could use some help wherever possible.

Thank you,

Smon

dsefs
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image
Erick Ramirez answered ·

As I stated in your other question (#9778), the stack trace indicates you have an authentication issue and that your app has not been provided with the correct credentials to access DSEFS.

A friendly note to let you know that this site is not intended to replace support issues and is designed for general questions.

We have limited ability to assist you in a Q&A forum so please log a ticket with DataStax Support so one of our engineers can assist you directly. You will need to provide the sample code that replicates the problem as well as a Diagnostics Report from OpsCenter to assist with the investigation. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

jaroslaw.grabowski_50515 avatar image
jaroslaw.grabowski_50515 answered ·

Hi!

Could you paste the full stack trace?

Are you able to connect to DSEFS with dsefs shell (`dse fs` command)? If no, could you paste the contents of `cat ~/.dsefs-shell.log` file?

Are you able to open spark-shell (`dse spark` command)? If no, could you paste the contents of `cat ~/.spark-shell.log` file?

Which DSE version is this?

Cheers,

Jarosław

9 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

@jaroslaw.grabowski_50515 Hi, Thank you for getting back to me. I am able to connect/shell into dsefs shell with dse fs and I am also able to shell into spark with dse spark which is why I am baffled that I am getting this error: dseError.txt


Also, I am using version 6.7.7

0 Likes 0 ·
dseerror.txt (2.6 KiB)

This is an authentication problem. What kind of authentication do you use?

0 Likes 0 ·
scano_183208 avatar image scano_183208 jaroslaw.grabowski_50515 ·

@jaroslaw.grabowski_50515

I am using the internal authentication that comes with DSE not KERBEROS or LDAP.


I am also using the superuser cassandra with the correct password in the app.

0 Likes 0 ·
Show more comments