question

vkayanala_42513 avatar image
vkayanala_42513 asked Erick Ramirez commented

Why is the consistency level changing from LOCAL_QUORUM to TWO?

Hi Support,

Our applications are configured to LOCAL_QUORUM on both reads and writes. Lately, we are noticing

com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency TWO (2 responses were required but only 1 replica responded, 1 failed). 

Not sure where this CL TWO coming from!

Along with this, noticing this below warn in system.log when CL TWO happened.

WARN [MessagingService-Incoming-/x.x.x.x] 2020-05-13 05:47:55,216 MessagingService.java:848 - Backpressure rejection while receiving WRITES.BATCH_STORE from /x.x.x.x

Thank You.

cassandra
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

yukim avatar image
yukim answered Erick Ramirez commented

Are you writing data with batch (BEGIN BATCH ... APPLY BATCH)?

When doing batch, Cassandra first writes batch log to several nodes in the cluster in case of failure. When you are writing data with CL of more than ONE, then batch log writing will use CL of TWO to replicate the batch log.

The error indicates it couldn't replicate batch log.


BTW if you are using Materialized View, updating MV uses batch as well, so that can be the case as well.

3 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image Erick Ramirez ♦♦ commented ·

Thanks for the clarification. I forgot to consider the failed batch log writes which makes sense from WriteFailureException which is returned by the coordinator when the replica write fails for batch writes.

0 Likes 0 ·
vkayanala_42513 avatar image vkayanala_42513 commented ·

@yukim We are doing single partition batches. I have read that "No batchlog mechanism is necessary for single partition batching". Source: https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useBatch.html


Our batch query is like this: BATCH of 1 (with 1 others) : INSERT INTO from_link (from_id,link_type,to_id) VALUES (?,?,?);

and not using MV's.

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ vkayanala_42513 commented ·

What does this mean?

BATCH of 1 (with 1 others)
0 Likes 0 ·