question

gpss1979 avatar image
gpss1979 asked smadhavan commented

Driver Timeout Exception

Hi All,

I have a few questions regarding the Java driver configuration and the below DriverTimeoutException that I fail to understand:

com.datastax.oss.driver.api.core.DriverTimeoutException Query timed out after 2 seconds.

As I understand, the default timeout of the driver is 2 seconds, whereas the default read timeout of the coordinator is 5 seconds. Shouldn't the driver have a timeout of at least 5 seconds as well?
To be more precise, what happens if the coordinator provides the data after 4 seconds for example?

Also, could it be that the driver gets a DriverTimeoutException because of excessive GC behavior or a high CPU load on the client itself, regardless of Cassandra or a specific query?


Thanks a lot!

java driver
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

bettina.swynnerton avatar image
bettina.swynnerton answered smadhavan commented

Hi @gpss1979,

the following page from the driver documentation should help to clarify this further:

https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/retries/#on-read-timeout

The DriverTimeoutException happens when the coordinator doesn't reply to the client in time.

The coordinator returns a READ_TIMEOUT error if the responding replicas take more than the allowed time (defined as read_request_timeout_in_ms in the cassandra.yaml) to respond back to the coordinator.

While it's possible that the client would be affected by excessive GC pauses on the client node, it is not something that I have seen. I would first look in more detail into the cluster state when these DriverTimeoutExceptions occur.

I hope this helps.

2 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

gpss1979 avatar image gpss1979 commented ·

The cluster state is fine during those exceptions and read latency is at most 5ms. The issue happens on more than one environment, so I think it's less related to the Cassandra cluster, but more on the client's end.
I didn't find any network related issues, so that's why I'm assuming some sort of odd client behavior. Increasing the timeout mitigates this issue and that's a good reason to believe that the client might be overloaded and/or queuing requests. But not sure that in this scenario, I should expect a DriverTimeoutException Exception.

The issue started to appear after upgrading the driver from v3.1 to v4.8. my assumption is that on 3.1, the default timeout was set to 12 seconds if I'm not mistaken and in the newer version it's set to 2 seconds.

I'd appreciate any ideas you may have, because I'm a bit lost here.

Thanks a bunch!

[Posted as a comment since it's not an "answer"]

0 Likes 0 ·
smadhavan avatar image smadhavan ♦ gpss1979 commented ·

@gpss1979, see if the answer that was posted earlier helps you in this case.

0 Likes 0 ·