question

francisdomoney@aol.com avatar image
francisdomoney@aol.com asked Erick Ramirez commented

Is this useful for debugging week 5 homework on Ubuntu Linux 20.04?

Feedback from trying to use the Kubernetes and Datastax Workshop on Ubuntu 20.04

Do let me know if this is any use

Section 1 Workshop Materials

1 Install Docker

Ubuntu doesn’t like yum

use https://linuxize.com/post/how-to-install-and-use-docker-on-ubuntu-20-04/

2 install Docker Compose

Ubuntu doesn’t like yum

Use https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-20-04

3 install Kubernetes command line tool

The curl files work fine. You may need to install curl first sudo apt update

sudo apt install curl

4 Install watch

This already exists on Ubuntu Linux

5 Install kind

The curl files download works

6 Create a Kind Cluster

The URL in “As of now the Cassandra Operator needs a worker per node. We will create a Kubernetes cluster based on the following configuration 1 master, 5 workers “ is defective. It omits “setup-your-cluster”

You need to download all the directories from the setup your cluster and install them below the directory you are operating from

The rest of 6 works OK

7 Create namespace and storageClass

Works as described

Section 2 Deploy a Cassandra Cluster with cass-operator

1 Install Cass Operator

Works as described

2 Create a single node cluster

Works as described

3 Browse your cluster

Works as described

4 Scale up our cluster

Skip from here as far as 4d as memory limitations will kill your machine

skip as far as Executing the command to login to a pod.

5 Executing CQL

This works as far as step “change the replication factor of killrvideo1

Change the replication factor of killrvideo and changing the consistency level caused the cql to return an error HOST NOT FOUND

6 Change the Cassandra configuration

Don’t upgrade to three clusters. It will kill your machine

Section 3 Metrics Management Visualisation

1 Clean up cluster defined in previous sections

Works as described

2 Install Operator Lifecycle Manager (OLM)

Works as described

3 Install Prometheus Operator

Works as described

4 Install Grafana Operator

Works as described

5 Start DSE Cluster

Works as described

6 Browse DSE Cluster

Works as far as Enter the Node and Show status

dsetool status dies.

kubectl -n cass-operator exec -it cluster2-dc2-default-sts-0 -- /bin/bash

  1. Defaulting container name to cassandra.

  2. Use 'kubectl describe pod/cluster2-dc2-default-sts-0 -n cass-operator' to see all of the containers in this pod.

  3. dse@cluster2-dc2-default-sts-0:~$ dsetool status

  4. Connection refused to host: 127.0.0.1; nested exception is:

  5. java.net.ConnectException: Connection refused (Connection refused)

7 Blow your mind

Not attempted.

8 Stop and clean up

Works as described.

You may need to restart your machine to clear any stray connections.

workshopcass-operator
1 comment
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

francisdomoney@aol.com avatar image francisdomoney@aol.com commented ·

This needs some example output so we know if we are seeing real logs or if something has gone wrong.

5c.Have a look in the logs of a pod

kubectl -n cass-operator logs cluster2-dc2-default-sts-0 -c cassandra
0 Likes 0 ·

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

Thanks for sharing this. Allow me to provide some friendly feedback:

  • Admins know that YUM is the package manager for CentOS/RHEL distributions and systems running Debian distributions use APT.
  • If you choose to run this on your own machine, you need a machine with at least 16GB of RAM. You are installing a database package after all.
  • We don't recommend doing the metrics exercise on your machine unless you have 32GB of RAM or more -- DSE requires more resources since it includes several components built on top of just a Cassandra instance.

Cheers!

3 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

David Jones-Gilardi avatar image David Jones-Gilardi ♦ commented ·

Wow, thank you @francisdomoney@aol.com for taking notes through all of the sections for Ubuntu. As far as the issues you ran into like:

Change the replication factor of killrvideo and changing the consistency level caused the cql to return an error HOST NOT FOUND

AND

dse@cluster2-dc2-default-sts-0:~$ dsetool status
Connection refused to host: 127.0.0.1; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)

I am almost 100% sure these are due to cases where all of the nodes have not completed their bootstrap. Ensure that your k8s pod status displays 2/2. If you see 1/2 even though the status might say "running" it is not complete.



0 Likes 0 ·
francisdomoney@aol.com avatar image francisdomoney@aol.com commented ·

Thanks Eric. That is really useful to know.

It illustrates the problem of trying to offer a solution that works on all the systems worldwide. It is not just windows that causes problems.

I talked to my local supplier yesterday and upgrading to 32GB won't break the bank.


For me to try and run a similar course in North Africa I will need to build a Local Cloud Server cluster to allow people to connect.


Trying to run anything in North Africa on commercial clouds is, in local terms, prohibitively expensive. $50 in US terms is a small amount. For a student in the Maghreb it is the food budget for a month.

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ francisdomoney@aol.com commented ·

We completely understand which is why we offer the courses and workshops for free. But at the scale that we're running it, provisioning 2,000+ instances for attendees gets very expensive very quickly costing thousands of dollars every hour. :)

0 Likes 0 ·