I have upgraded from DSE 3.5 to 4.9to my product. Have noticed dip in microservices capabilities. the tps is down by 15 percent. after a certain level getting
com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S at com.datastax.oss.driver.api.core.DriverTimeoutException.copy(DriverTimeoutException.java:34) at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53) at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProces
Increased the timeout to 30 sec experimentally from 2 sec default.
then the timeout vanishes as expected but server at times giving service unavailable 503 to the client.Seems something is oveburdening it.
Driver configuration:
final DriverConfigLoader configLoader = DriverConfigLoader.programmaticBuilder() .withString(DefaultDriverOption.REQUEST_CONSISTENCY, ConsistencyLevel.LOCAL_QUORUM.name()) .withString(DefaultDriverOption.RECONNECTION_POLICY_CLASS, ExponentialReconnectionPolicy.class.getName()) .withInt(DefaultDriverOption.RECONNECTION_BASE_DELAY, baseReconnectionIntervalSec * 1000) .withInt(DefaultDriverOption.RECONNECTION_MAX_DELAY, maxReconnectionIntervalSec * 1000) .withDuration(DefaultDriverOption.REQUEST_TIMEOUT, Duration.ofSeconds(30))//Test changes for driver timeout .withString(DefaultDriverOption.RETRY_POLICY_CLASS, CassandraCustomRetryPolicyXXXX.class.getName()).build();