question

meet.anadkat_163766 avatar image
meet.anadkat_163766 asked Erick Ramirez edited

How can I get the status of Spark applications in DSE 5.0?

Hello,

I am running DSE 5.0 version with analytics enabled. My spark applications are running fine in client as well as cluster modes. However, if I try to get the status of an application that is running in cluster deploy-mode, it gives me following error:

Error: Requesting submission statuses is only supported in standalone or Mesos mode!

After a bit research read somewhere that DSE uses custom cluster manager. If that's the case, how come dse --help is showing the option to run status commands.

And if so, is there any other way to get the status of submitted applications?

Thanks in advance!

dseanalytics
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

@meet.anadkat_163766 there appears to be a bug with the dse spark-submit --status which does not work with the DSE Resource Manager. We no longer provide fixes since DSE 5.0 has reached EOL so the only workaround I can give is to check the status of applications on the Spark master web UI. Cheers!

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.