Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

yariv.amar_82168 avatar image
yariv.amar_82168 asked ·

NoHostAvailableException due to BusyPoolException

hi
on my testing environment, i sometimes get the following exception during CI:

com.datastax.driver.core.exceptions.BusyPoolException: [vm-test-dse-cass-] 
Pool is busy (no available connection and the queue has reached its max size 256

i've tried to increase the pool size and connection settings on the driver but w/o any luck. moreover, no matter how i change the driver settings, eg. set poolingOptions.maxQueueSize to 1 or 512, the result is the same error "max size 256"...
questions:
1. what cause BusyPoolException?

2. how can I overcome this? are there any settings on the server side?

3. is it a bug in the driver (cassandra-driver-3.6.0)?
thank you

java driver
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

This kind of BusyPoolException gets thrown when all the connections in the pool is used and there are no available slots left:

com.datastax.driver.core.exceptions.BusyPoolException: [vm-test-dse-cass-] 
Pool is busy (no available connection and the queue has reached its max size 256)

This can happen when:

  • the application is sending multiple long-running requests that eventually tie up all the connections in the pool until there are none left, or
  • the nodes are overloaded and requests are queueing up.

You need to review your queries and make sure you are only doing single-partition requests. You also need to throttle the traffic to make sure you are not overloading the cluster.

You mentioned that you've set different values for poolingOptions.maxQueueSize but the exception still reports "max size 256". This indicates to me that you haven't recompiled your app or perhaps you're running an old version of your app. Cheers!

1 comment Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

yariv.amar_82168 avatar image yariv.amar_82168 ·

thank you.
the CI pipeline first compiles the code and then run the tests (which indeed results in lot's of queries). so the changes in maxQueueSize should have been used.

Are there any server side configuration that can help in mitigation (or monitoring, that will give a clue)?


0 Likes 0 · ·