question

dshevchuk avatar image
dshevchuk asked Erick Ramirez edited

DSE Analytics - using separate node as driver node

I have a DSE 6.8 cluster with two DC - Cassandra nodes and Analytics Solo nodes.

Also I have a separate node where OpsCenter is installed and it's Lifecycle manager was used for cluster installation.

I am planning to use this OpsCenter node as Spark driver node - for submitting Spark applications. I want to have here installed Livy/job-server and also Zeppelin.

So, I need to use "dse exec" command to run all this third-party software. But "dse" package is not installed at this node yet.

My question is: will I do this right if I perform next steps:

1. Will install "dse: package at OpsCenter node.

2. Will disable "dse" service at OpsCenter node (systemctl stop dse && systemctl disable dse)

3. Will export dse configuration from existing node and import to OpsCenter node.

4. Finally use "dse exec" at OpsCenter node to launch software I need.

I am in doubt if using separate node is a right way for using Livy/Job-server and Zeppelin. With implementing step 2 at OpsCenter node I will have a separate dse node that will be not used. Can this impact somehow my cluster? Are there any guidelines or things I should keep in mind?

analytics
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

bettina.swynnerton avatar image
bettina.swynnerton answered bettina.swynnerton edited

Hi @dshevchuk,

you do not need to install DSE as a package on the OpsCenter node.

Downloading and extracting a DSE 6.8 tarball should be sufficient, this will give you access to the dse command (i.e in /Downloads/dse-6.8.4/bin) .

For the download: https://downloads.datastax.com/#enterprise

You can then use dse client-tool on the OpsCenter node to import the configuration from your Spark DC. DSE should not be running on the OpsCenter node.

This should have no impact on your cluster and is the recommended way to launch Spark command from a client against a remote DSE cluster.

https://docs.datastax.com/en/dse/6.8/dse-dev/datastax_enterprise/spark/sparkRemoteCommands.html

This should also give you access to the dse exec capability, but I admit that this is not something I have tried.

Let us know how you get on.

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.