Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

whpw avatar image
whpw asked Erick Ramirez answered

What are steps for recovering a node after the data volume has failed?

Hi, what are the right steps if the data volume break because of a disk failure? The Cassandra binaries and config files are available only the data are corrupt. My approach would be after the data volume is repaired and cleaned (new filesystem) to run a full repair on the affected node. Is this the right way? or what must be done so that the node is recognized as faulty and no read and write operations are made? Many thanks in advance.

replace nodes
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

Running a repair will not work if the data directory is empty. When you start the node, it will bootstrap itself into the cluster as if it's a brand new node.

The correct way to recover a node from a failed disk is to bootstrap it with the "replace address" flag as described in Replacing a dead node. The process involves adding the replace_address_first_boot JVM option so the node will "replace itself" and take over the same token(s) as the original node.

Make sure the node is not listed as a seed node in its own cassandra.yaml otherwise no data will stream to it from other nodes in the cluster and it will immediately join the cluster with no data. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.