victor.lopezselva_180476 avatar image
victor.lopezselva_180476 asked Erick Ramirez edited

What is the optimal concurrency level to use with the Python driver?

Recently I found out about cassandra.concurrent available on the Python driver for Cassandra. On the docs, it says that if `protocol_version` si 3 or higher (which is our case) one can "safely experiment with higher levels of concurrency". I was wondering how we can find out what's our optimal value of `concurrency`.

With our protocol version we can't select how many connections we create, because the driver it uses connection pooling, so how can we find out the best value for this parameter?

10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

therealfalcon avatar image
therealfalcon answered victor.lopezselva_180476 commented

For v3 and higher, there is only ever a single connection to any server host, but up to 32768 (currently hardcoded) queries can be in flight at at time. To figure out the optimal value, you can use the instructions here:

The optimal concurrency level depends on both the client and server hardware specifications as well as other factors like:

  • the server cluster size
  • the number of instances of the application accessing the database
  • the complexity of the queries

When implementing an application, launch a fixed number of asynchronous operations using the concurrency level as the maximum. As each operation completes, add a new one. This ensures your application's asynchronous operations will not exceed the concurrency level.

1 comment Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

victor.lopezselva_180476 avatar image victor.lopezselva_180476 commented ·

Thanks @therealfalcon! This is very useful!

0 Likes 0 ·