Getting the following error when using livy client to dowload a jar file from dsefs directory to a tmp folder.
When I sudo su or simply use dse fs I do not get any errors.
20/12/28 22:06:45 INFO BatchSession: Batch session 19 created [appid: null, state: running, info: {driverLogUrl=null, sparkUiUrl=null}] 20/12/28 22:06:55 INFO LineBufferedStream: Exception in thread "main" com.datastax.bdp.fs.client.DseFsConnectionException: Could not connect to DSEFS. Last endpoint tried: 192.168.21.4:5598 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.client.RestClientProxy.<init>(RestClientProxy.scala:161) 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.client.DseFsClient.liftedTree1$1(DseFsClient.scala:62) 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.client.DseFsClient.<init>(DseFsClient.scala:62) 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.hadoop.DseFileSystem.initialize(DseFileSystem.scala:129) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:362) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.SparkSubmit$.downloadFile(SparkSubmit.scala:885) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmit$$anonfun$prepareSubmitEnvironment$1.apply(DseSparkSubmit.scala:290) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmit$$anonfun$prepareSubmitEnvironment$1.apply(DseSparkSubmit.scala:290) 20/12/28 22:06:55 INFO LineBufferedStream: at scala.Option.map(Option.scala:146) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmit$.prepareSubmitEnvironment(DseSparkSubmit.scala:290) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmit$.submit(DseSparkSubmit.scala:140) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmit$.main(DseSparkSubmit.scala:106) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmitBootstrapper$.main(DseSparkSubmitBootstrapper.scala:109) 20/12/28 22:06:55 INFO LineBufferedStream: at org.apache.spark.deploy.DseSparkSubmitBootstrapper.main(DseSparkSubmitBootstrapper.scala) 20/12/28 22:06:55 INFO LineBufferedStream: Caused by: com.datastax.bdp.fs.rest.UnknownServerResponseException: The server sent an error response this client did not understand: HTTP 401: 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.rest.RestResponse$stateMachine$macro$331$1.apply(RestResponse.scala:51) 20/12/28 22:06:55 INFO LineBufferedStream: at com.datastax.bdp.fs.rest.RestResponse$stateMachine$macro$331$1.apply(RestResponse.scala:44) 20/12/28 22:06:55 INFO LineBufferedStream: at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) 20/12/28 22:06:55 INFO LineBufferedStream: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) 20/12/28 22:06:55 INFO LineBufferedStream: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) 20/12/28 22:06:55 INFO LineBufferedStream: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:465) 20/12/28 22:06:55 INFO LineBufferedStream: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) 20/12/28 22:06:55 INFO LineBufferedStream: at java.lang.Thread.run(Thread.java:748) 20/12/28 22:07:00 WARN BatchSession$: spark-submit exited with code 1 20/12/28 22:07:00 ERROR SparkProcApp: spark-submit exited with code 1