I have a batch job used to publish data on DSE Graph. The job keeps Cassandra session cached through its lifecycle and keep creating "GraphTraversal" objects as needed. In my test run of few hundred mutation traversals it worked fine, when I did execution of around 50k traversals, I see few of the traversals failing intermittently with following error:
com.datastax.oss.driver.api.core.NoNodeAvailableException: No node was available to execute the query at com.datastax.oss.driver.api.core.AllNodesFailedException.fromErrors(AllNodesFailedException.java:53) at com.datastax.dse.driver.internal.core.cql.continuous.ContinuousRequestHandlerBase.sendRequest(ContinuousRequestHandlerBase.java:366) at com.datastax.dse.driver.internal.core.cql.continuous.ContinuousRequestHandlerBase.onThrottleReady(ContinuousRequestHandlerBase.java:276) at com.datastax.oss.driver.internal.core.session.throttling.PassThroughRequestThrottler.register(PassThroughRequestThrottler.java:52) at com.datastax.dse.driver.internal.core.graph.ContinuousGraphRequestHandler.<init>(ContinuousGraphRequestHandler.java:90) at com.datastax.dse.driver.internal.core.graph.GraphRequestAsyncProcessor.process(GraphRequestAsyncProcessor.java:83) at com.datastax.dse.driver.internal.core.graph.GraphRequestAsyncProcessor.process(GraphRequestAsyncProcessor.java:38) at com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230) at com.datastax.dse.driver.api.core.graph.GraphSession.executeAsync(GraphSession.java:82)
We are building the traversals using Fluent APIs and then running it explicit using async execution of DSE driver.
com.datastax.dse.driver.api.core.graph.GraphSession#executeAsync
code snippet:
CompletionStage<AsyncGraphResultSet> res = cassandraConnector.getCqlSession().executeAsync(FluentGraphStatement.newInstance(traversal)); res.whenComplete((r, t) -> { if (t != null) { logger.error("error occured while executing traversals.", t); reportError(t); } });
Here is my driver configuration:
datastax-java-driver { basic { contact-points=["myhost:9042"] load-balancing-policy.local-datacenter = GraphDC1 session-name = cfx-adv request.page-size = 50000 config-reload-interval = 0 } basic.graph{ name=iag_test traversal-source="g" timeout = 60 seconds } advanced { protocol.compression = lz4 connection { max-requests-per-connection = 1024 pool { local.size = 1 remote.size = 1 } } }#end of advanced }
DSE Version: 6.8.1, Driver : 4.8
Can somebody help me identify why driver is not able to find the node while service is up and running. Is it something related to driver configuration or thread pool ?