Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

mishra.anurag643_153409 avatar image
mishra.anurag643_153409 asked Erick Ramirez edited

Can nodes be overloaded if read consistency is LOCAL_ONE?

I am reading data from cassandra table with the help of pyspark , and read consistency at cassandra level it is local_one . I am observing nodes are becoming unresponsive and sometimes throwing an error no replica is available . This is only happening when pyspark job is trying to read cassandra table .

I have below query:

as per my understanding read consistency local_one in this case read request should be served fast as only one closest replica needs to respond in this case . But is it possible cassandra node might be overloaded in case read consistency with local_one ?

spark-cassandra-connector
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

jaroslaw.grabowski_50515 avatar image
jaroslaw.grabowski_50515 answered

Yes, it's possible overwhelm a node with a Spark job. Use read/write throttling to avoid it https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md#read-tuning-parameters.

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez edited

If nodes are capable of handling a maximum of 10,000 operations per second, then that is all they can handle.

If your app tries to read at 12K or 15K ops/s, it doesn't matter if you read at a consistency level of LOCAL_ONE. You will still overload the nodes because you're sending requests higher than their maximum capacity. Cheers!

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

mishra.anurag643_153409 avatar image mishra.anurag643_153409 ·

[Follow up question posted in #10296]

0 Likes 0 ·