Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

vraghavach avatar image
vraghavach asked Erick Ramirez edited

MCAC reporting Cassandra down when it is not

Here is the environment:

  • Cassandra: 4.0.3 - running on bare metal in Digital Ocean
  • Nodes: 1
  • Prometheus: running under Kubernetes in Digital Ocean
  • OS: CentOS
  • Java: 1.8
  • Configuration files: No major changes. Prett

Datastax MCAC has been installed and from the system.log file it shows that it is reporting the metrics.

However despite Cassandra being up and running I get alerts from Prometheus every 3 hours stating it is down.

Any help would be appreciated.

metrics collector
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez edited

If you haven't already, I'd suggest you review the Prometheus events leading up to the alert getting generated. I'd also recommend you check the operating system logs for clues. If the node is overloaded, there's a possibility that the agent is not getting enough CPU cycles and it might be the reason you're getting the alerts.

You can also have a look at the datalog events to help you troubleshoot the problem. There is a script included with MCAC that parses the logs. Details are in the FAQ section of the README on GitHub. Cheers!

2 comments Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks Erick for the details. Will review and report back.

0 Likes 0 ·

Erick, Figured it out. It was my mistake. I had configured the MCAC port (9103) incorrectly in the Kubernetes universe. All good now.

0 Likes 0 ·