Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

mishra.anurag643_153409 avatar image
mishra.anurag643_153409 asked ·

If consistency is LOCAL_QUORUM and RF=3, does Spark connect to two nodes in the cluster?

[FOLLOW UP QUESTION TO #10306]

when consistency level is set to LOCAL_QUORUM with rf = 3 , in this case does spark connect two nodes to the cassandra cluster or every client request is only served by coordinator , hence coordinator reads data from one of replica and that data is then forwarded to spark by coordinator?

spark-cassandra-connector
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

The Spark connector uses the Java driver under the hood. When a read is requested by the connector, the driver connects to a node in the cluster which acts as the coordinator for the request.

When the consistency is set to LOCAL_QUORUM and the keyspace has a replication factor of 3 for the local DC, the coordinator contacts 2 replicas (quorum of 3) to request the data. Provided the responses are consistent between the 2 replicas, the result is returned to the Java driver and eventually the connector as the client.

Note that I have provided a simplified version of events in my answer for brevity. For more information, see How read requests are accomplished in Cassandra. Cheers!

2 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

I am concerned with :

1. does spark connect to co-ordinator only for read or co-ordinator route spark to connect with replica , I am asking this question as I have concern how number of requests served by cassandra cluster ?

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ mishra.anurag643_153409 ·

The connector itself doesn't connect to the cluster. As I stated previously, it uses the Java driver to connect to the cluster. Think of the connector as just another client just like cqlsh or any other application. Cheers!

P.S. cqlsh is a standalone client/app which uses another driver (Python) to connect to a Cassandra cluster.

0 Likes 0 ·