Recently I found out about cassandra.concurrent available on the Python driver for Cassandra. On the docs, it says that if `protocol_version` si 3 or higher (which is our case) one can "safely experiment with higher levels of concurrency". I was wondering how we can find out what's our optimal value of `concurrency`.
With our protocol version we can't select how many connections we create, because the driver it uses connection pooling, so how can we find out the best value for this parameter?
For v3 and higher, there is only ever a single connection to any server host, but up to 32768 (currently hardcoded) queries can be in flight at at time. To figure out the optimal value, you can use the instructions here:
The optimal concurrency level depends on both the client and server hardware specifications as well as other factors like:
- the server cluster size
- the number of instances of the application accessing the database
- the complexity of the queries
When implementing an application, launch a fixed number of asynchronous operations using the concurrency level as the maximum. As each operation completes, add a new one. This ensures your application's asynchronous operations will not exceed the concurrency level.
6 People are following this question.
