question

pramod.dba31_92912 avatar image
pramod.dba31_92912 asked Erick Ramirez answered

Failed to execute method DseResourceManager.heartbeat java.lang.RuntimeException: Failed to execute query

What does this stack trace indicate:

INFO  [IOThread-6] 2021-02-13 19:10:03,154  RpcMethod.java:199 - Failed to execute method DseResourceManager.heartbeat
java.lang.RuntimeException: Failed to execute query
        at com.datastax.bdp.router.InternalQueryRouter.withQueryExceptionsHandled(InternalQueryRouter.java:117)
        at com.datastax.bdp.util.rpc.RpcMethod.execute(RpcMethod.java:173)
        at com.datastax.bdp.cassandra.cql3.RpcCallStatement.execute(RpcCallStatement.java:107)
        at org.apache.cassandra.cql3.QueryProcessor.lambda$processStatement$4(QueryProcessor.java:266)
        at io.reactivex.internal.operators.single.SingleDefer.subscribeActual(SingleDefer.java:36)
        at io.reactivex.Single.subscribe(Single.java:2700)
        at org.apache.cassandra.utils.flow.RxThreads$1SubscribeOn.lambda$subscribeActual$0(RxThreads.java:50)
        at org.apache.cassandra.concurrent.TPCRunnable.run(TPCRunnable.java:68)
        at org.apache.cassandra.concurrent.IOScheduler$PooledTaskWorker.run(IOScheduler.java:319)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The channel has been terminated.
        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
        at com.datastax.bdp.util.rpc.RpcMethod.lambda$execute$8(RpcMethod.java:174)
        at com.datastax.bdp.router.InternalQueryRouter.withQueryExceptionsHandled(InternalQueryRouter.java:94)
        ... 16 common frames omitted
Caused by: java.io.IOException: The channel has been terminated.
        at com.datastax.bdp.node.transport.ClientServerConnection.terminatePendingRequests(ClientServerConnection.java:286)
        at com.datastax.bdp.node.transport.ClientServerConnection.access$800(ClientServerConnection.java:60)
        at com.datastax.bdp.node.transport.ClientServerConnection$ChannelStateHandler.channelInactive(ClientServerConnection.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
        at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947)
        at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:839)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:465)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
        ... 2 common frames omitted


dsespark
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

For whatever reason, the connection with one of the nodes got terminated but the stack trace you posted doesn't indicate why.

Please log a ticket with DataStax Support and attach a Diagnostics Report from OpsCenter so one our engineers can assist you. Cheers!

Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.