Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

victor.lopezselva_180476 avatar image
victor.lopezselva_180476 asked ·

What is the optimal concurrency level to use with the Python driver?

Recently I found out about cassandra.concurrent available on the Python driver for Cassandra. On the docs, it says that if `protocol_version` si 3 or higher (which is our case) one can "safely experiment with higher levels of concurrency". I was wondering how we can find out what's our optimal value of `concurrency`.

With our protocol version we can't select how many connections we create, because the driver it uses connection pooling, so how can we find out the best value for this parameter?

driverpythonconcurrency
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

therealfalcon avatar image
therealfalcon answered ·

For v3 and higher, there is only ever a single connection to any server host, but up to 32768 (currently hardcoded) queries can be in flight at at time. To figure out the optimal value, you can use the instructions here:

https://docs.datastax.com/en/devapp/doc/devapp/driverManagingConcurrency.html#Limitingsimultaneousrequestsinyourapplicationcode

The optimal concurrency level depends on both the client and server hardware specifications as well as other factors like:

  • the server cluster size
  • the number of instances of the application accessing the database
  • the complexity of the queries

When implementing an application, launch a fixed number of asynchronous operations using the concurrency level as the maximum. As each operation completes, add a new one. This ensures your application's asynchronous operations will not exceed the concurrency level.

1 comment Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks @therealfalcon! This is very useful!

0 Likes 0 · ·