Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

lziegler avatar image
lziegler asked Erick Ramirez edited

Reworking Spark Client Mode Jobs as Clustermode - options to track job completion?

Attempting to reengineer several client mode dse spark jobs as cluster mode. What method is most foolproof to tell when the cluster mode job has completed from a shell script running locally on the data node?

  1. dse spark-submit --status driver-id ... loop
  2. rest api call such as ... curl -L http://nodename:18080/api/v1/applications?status=running

Any other recommendations? Thank you!

analytics
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

That's pretty much it. There aren't any features added in DSE for this purpose. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.