Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

rajib76 avatar image
rajib76 asked Erick Ramirez answered

Read repairs get triggered on every read request

We have noticed that a read repair is initiated on every select query. Since read repair is blocking, it increases our read latency. Is there anyway to optimize this behavior. Can we tell the selects not to initiate a read repair?

read-repair
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

Read repairs are one of the mechanisms that Cassandra implements to keep replicas consistent.

When Cassandra detects inconsistencies reading data with a consistency higher than ONE or LOCAL_ONE, a read repair gets triggered to repair the data on the inconsistent replica(s). To satisfy the required consistency, Cassandra does not return the result until the replicas are consistent so the request is blocked until the operation has completed successfully.

For example reading from a keyspace with a replication factor of 3 and consistency LOCAL_QUORUM, the coordinator node will request data from 2 replicas (quorum of 3 nodes). The coordinator will compare the result from the replicas and if they don't match, the coordinator will repair the bad replica. When the repair has completed, the result is returned to the client. For more info, see Read Repair: repair during read path.

There is no way to disable this behaviour since the consistency level of the request determines how many replicas must "agree" for the result to be valid.

The real issue here is that your cluster is inconsistent because replicas are missing writes. Nodes drop mutations (INSERTs, UPDATEs, DELETEs) when they get overloaded with writes because the commitlog disk cannot keep up. You should consider throttling the app requests or increase the capacity of your cluster by adding nodes. Cheers!

Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.