Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

nagasree963_193876 avatar image
nagasree963_193876 asked ·

Cassandra pod is failing with liveness and readiness

I am deploying cassandra operator in GCP. But it is getting like readiness and liveness probe failed..

$ kubectl -n cassandra-aiops get pods
NAME                                   READY STATUS    RESTARTS AGE
cass-operator-1-1610454343-cmd-1-nhft6 0/1   Completed 0        74m
cass-operator-7b4847f644-p6twj         1/1   Running   0        73m
cluster1-dc1-default-sts-0             1/2   Running   0        72m
cluster1-dc1-default-sts-1             1/2   Running   0        46m
cluster1-dc1-default-sts-2             1/2   Running   0        72m

when I describe the pod, I can see as below:

Containers:
cassandra:
Container ID: docker://5cc073142aaca5585801161960dd7b0803bf2d8c656c7609fafed39bd109b894
Image: gcr.io/saaspoc-gtso-enterprise-gke/gkegtsoent/cassandra-operator/cassandra-mgmtapi-3_11_6:0.1.0
Image ID: docker-pullable://gcr.io/saaspoc-gtso-enterprise-gke/gkegtsoent/cassandra-operator/cassandra-mgmtapi-3_11_6@sha256:25611452cec30bc9bf8ac734f6258525e905612f2f5874c62023c4983f256dc1
Ports: 9042/TCP, 8609/TCP, 7000/TCP, 7001/TCP, 8080/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
State: Running
Started: Tue, 12 Jan 2021 12:28:41 +0000
Ready: False
Restart Count: 0
Limits:
cpu: 500m
memory: 1Gi
Requests:
cpu: 500m
memory: 1Gi
Liveness: http-get http://:8080/api/v0/probes/liveness delay=15s timeout=1s period=15s #success=1 #failure=3
Readiness: http-get http://:8080/api/v0/probes/readiness delay=20s timeout=1s period=10s #success=1 #failure=3
Environment:

Can anyone help me on this

cass-operator
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

In my experience, this problem often occurs (but not always) because of lack of resources.

Consider adding more RAM to Docker. 8GB is the minimum but 16GB is recommended.

If the cluster is not memory-bound, review the Cassandra logs in each pod for clues as to why it is unable to start. For the record, I'm not asking for the logs. I'm just providing some troubleshooting suggestions. Cheers!

2 comments Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

nagasree963_193876 avatar image nagasree963_193876 ·

Thank you , it worked for me

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ nagasree963_193876 ·

That's great. Thanks for letting me know. Cheers!

0 Likes 0 ·