question

latifah.pres_180632 avatar image
latifah.pres_180632 asked pmcfadin published

docker not installing agent

I'm using docker to install a dse-server and opscenter. After choosing manage existing cluster and adding the IPAddress, I then click install agent manually. The docs state that agents are installed already when using docker. Once ops center connect it shows that the node is unknown and no agent was set. No sure what's going on.

dseopscenterdocker
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

latifah.pres_180632 avatar image
latifah.pres_180632 answered pmcfadin published

dse version: 6.7.4

opscenter version:

I didn't do a custom build. I downloaded docker desktop and pulled straight from the hub

docker exec -it my-dse bash -c 'ps -ef'
UID        PID  PPID  C STIME TTY          TIME CMD
dse          1     0 44 16:40 ?        00:02:19 /docker-java-home/bin/java -Ddse
dse        135     1 12 16:40 ?        00:00:38 [java] <defunct>
dse        947     1  3 16:41 ?        00:00:09 collectd_wrapper /opt/dse/resour
dse       1028     1  0 16:41 ?        00:00:00 [iostat] <defunct>
dse       1029     1  0 16:41 ?        00:00:00 [iostat] <defunct>
dse       1213     0  0 16:45 pts/0    00:00:00 ps -ef


docker exec -it my-dse bash -c 'cat /opt/agent/conf/address.yaml'
conf/address.yaml'
stomp_interface: 172.17.0.2
use_ssl: 0
local_interface: 172.17.0.3
hosts: ["172.17.0.3"]
cassandra_install_location: /opt/dse
cassandra_log_location: /var/log/cassandra


docker exec -it --user root my-dse cat /entrypoint.sh
#!/bin/bash
# -*- mode: sh -*-
#
# Copyright DataStax, Inc, 2017
#   Please review the included LICENSE file for more information.
#

set -e

. /base-checks.sh

link_external_config "${DSE_HOME}"

#create directories for holding the node's data, logs, etc.
mkdir -p /var/lib/spark/worker
mkdir -p /var/lib/spark/rdd
mkdir -p /var/log/spark/worker
mkdir -p /var/log/spark/master

############################################
# Set up variables/configure the image
############################################

IP_ADDRESS="$(hostname --ip-address)"
CASSANDRA_CONFIG="${DSE_HOME}/resources/cassandra/conf/cassandra.yaml"
CASSANDRA_RACK_CONFIG="${DSE_HOME}/resources/cassandra/conf/cassandra-rackdc.properties"

# SNITCH sets the snitch this node will use. Use GossipingPropertyFileSnitch if not set
: ${SNITCH=GossipingPropertyFileSnitch}

# NATIVE_TRANSPORT_ADDRESS is where we listen for drivers/clients to connect to us. Setting to 0.0.0.0 by default is fine
# since we'll be specifying the NATIVE_TRANSPORT_BROADCAST_ADDRESS below
: ${NATIVE_TRANSPORT_ADDRESS='0.0.0.0'}

# LISTEN_ADDRESS is where we listen for other nodes who want to communicate. 'auto' is not a valid value here,
# so use the hostname's IP by default
: ${LISTEN_ADDRESS='auto'}
if [ "$LISTEN_ADDRESS" = 'auto' ]; then
    LISTEN_ADDRESS="$IP_ADDRESS"
fi

# BROADCAST_ADDRESS is where we tell other nodes to communicate with us. Again, 'auto' is not a valid value here,
# so default to the LISTEN_ADDRESS or the hostname's IP address if set to 'auto'
: ${BROADCAST_ADDRESS="$LISTEN_ADDRESS"}
if [ "$BROADCAST_ADDRESS" = 'auto' ]; then
    BROADCAST_ADDRESS="$IP_ADDRESS"
fi

# By default, tell drivers/clients to use the same address that other nodes are using to communicate with us
: ${NATIVE_TRANSPORT_BROADCAST_ADDRESS=$BROADCAST_ADDRESS}

# SEEDS is for other nodes in the cluster we know about. If not set (because we're the only node maybe), just
# default to ourself
: ${SEEDS:="$BROADCAST_ADDRESS"}

# modify cassandra.yaml only if not linked externally
if should_auto_configure "$CASSANDRA_CONFIG" ; then
    echo "Applying changes to $CASSANDRA_CONFIG ..."

    sed -ri 's/(endpoint_snitch:).*/\1 '"$SNITCH"'/' "$CASSANDRA_CONFIG"

    # Replace the default seeds setting in cassandra.yaml
    sed -ri 's/(- seeds:).*/\1 "'"$SEEDS"'"/' "$CASSANDRA_CONFIG"

    # Update the following settings in the cassandra.yaml file based on the ENV variable values
    for name in \
        broadcast_address \
        native_transport_broadcast_address \
        cluster_name \
        listen_address \
        num_tokens \
        native_transport_address \
        start_native_transport \
        ; do
        var="${name^^}"
        val="${!var}"
        if [ "$val" ]; then
          sed -ri 's/^(# )?('"$name"':).*/\2 '"$val"'/' "$CASSANDRA_CONFIG"
        fi
    done
    echo "done."
fi

if should_auto_configure "$CASSANDRA_RACK_CONFIG" ; then
    echo "Applying changes to $CASSANDRA_RACK_CONFIG ..."
    for rackdc in dc rack; do
        var="${rackdc^^}"
        val="${!var}"
        if [ "$val" ]; then
            sed -ri 's/^('"$rackdc"'=).*/\1 '"$val"'/' "$CASSANDRA_RACK_CONFIG"
        fi
    done
    echo "done."
fi

[ -z "$OPSCENTER_IP" ] && OPSCENTER_IP=$(getent hosts opscenter | awk '{ print $1 }')
if [ ! -z "$OPSCENTER_IP" ]; then
    echo "Configuring agent to connect to OpsCenter (${OPSCENTER_IP}) "
    cat > "$DSE_AGENT_HOME/conf/address.yaml" <<EOF
stomp_interface: ${OPSCENTER_IP}
use_ssl: 0
local_interface: ${IP_ADDRESS}
hosts: ["${IP_ADDRESS}"]
cassandra_install_location: $DSE_HOME
cassandra_log_location: /var/log/cassandra
EOF

$DSE_AGENT_HOME/bin/datastax-agent

fi

# Run the command
echo "Running $@"
exec "$@"
Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

@brasco --Donnie Roberson-- avatar image
@brasco --Donnie Roberson-- answered @brasco --Donnie Roberson-- commented

@latifah.pres_180632
The agent is installed within the DSE container. To make this work correctly you have to link the opscenter container to the DSE container with --link which adds an entry in /etc/hosts with the ip of opscenter container and hostname of opscenter. There is a script that runs at startup which looks for the hostname opscenter and will configure the address.yaml and start the agent.

The steps would be

Start Opscenter

docker run -e DS_LICENSE=accept -d -p 8888:8888 --name my-opscenter datastax/dse-opscenter

Start DSE and link Opscenter

docker run -e DS_LICENSE=accept --link my-opscenter:opscenter --name my-dse -d datastax/dse-server

Get Ip of DSE container

docker inspect my-dse | grep '"IPAddress":

Open Opscenter -> manage existing cluster -> enter ipaddress -> install agents manually


17 comments Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

@brasco --Donnie Roberson-- avatar image @brasco --Donnie Roberson-- commented ·

What version of OpsCenter and DSE are you using? Did you build a custom image?

docker exec -it my-dse dse -v



What is the output of ?

docker exec -it my-dse bash -c 'ps -ef'
docker exec -it my-dse bash -c 'cat /opt/agent/conf/address.yaml'
docker exec -it --user root my-dse cat /entrypoint.sh


0 Likes 0 ·
latifah.pres_180632 avatar image latifah.pres_180632 @brasco --Donnie Roberson-- commented ·
docker exec -it my-opscenter bash -c 'ps -ef'
UID        PID  PPID  C STIME TTY          TIME CMD
opscent+     1     0 82 16:41 ?        00:01:28 /docker-java-home/bin/java -serv
opscent+   162     0  0 16:43 pts/0    00:00:00 ps -ef


0 Likes 0 ·
@brasco --Donnie Roberson-- avatar image @brasco --Donnie Roberson-- latifah.pres_180632 commented ·

I was looking for the output from the dse container. That is just from Opscenter.

0 Likes 0 ·
Show more comments
latifah.pres_180632 avatar image
latifah.pres_180632 answered latifah.pres_180632 edited

Those are the exact commands that I used. I copied your in just to be sure, but same problem. I just can't get the opscenter to connect to the node. I can make a keyspace and tables through the command line so the node is working fine. It's just ops center. Here's what it looks like


{
  "rack": "rack1",
  "agent_status": {
    "storage_cassandra": {
      "updated_at": null,
      "last_up": 0,
      "status": "unknown"
    },
    "last_seen": null,
    "version": null,
    "condition": "NO_AGENT_DETECTED",
    "install_status": {
      "error-message": null,
      "state": null
    },
    "jmx": {
      "updated_at": null,
      "last_up": 0,
      "status": "unknown"
    },
    "monitored_cassandra": {
      "updated_at": null,
      "last_up": 0,
      "status": "unknown"
    },
    "http": {
      "failure_reason": "The agent is not responding to HTTP requests: java.net.ConnectException: Connection refused ",
      "updated_at": 1573570610,
      "last_up": 0,
      "status": "down"
    },
    "stomp": {
      "updated_at": null,
      "last_up": 0,
      "status": "unknown"
    },
    "limitations": []
  },
  "name": "172.17.0.3",
  "agent_install_type": null,
  "dc": "dc1"
}



Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.