I am getting below error when running spark job on remote server but in same network.
[cluster8-nio-worker-2] WARN com.datastax.driver.core.RequestHandler - casstest.example.com/172.31.5.120:9042 replied with server error (java.util.ConcurrentModificationException), defuncting connection. [Executor task launch worker for task 5459] ERROR org.apache.spark.executor.Executor - Exception in task 3.0 in stage 336.0 (TID 5459) com.datastax.driver.core.exceptions.TransportException: [casstest.example.com/172.31.5.120:9042] Connection has been closed
in Cassandra logs getting below exceptions:
system.log:ERROR [CoreThread-0] 2020-12-07 01:20:07,779 Message.java:829 - Unexpected exception during request; channel = [id: 0x5b9c0eee, L:/172.31.5.120:9042 - R:/172.31.2.270:54864] system.log-java.util.ConcurrentModificationException: null system.log- at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909) system.log- at java.util.ArrayList$Itr.next(ArrayList.java:859) system.log- at org.apache.cassandra.transport.CBUtil.writeStringList(CBUtil.java:325) system.log- at org.apache.cassandra.transport.Message$ProtocolEncoder.makeFrame(Message.java:438) system.log- at org.apache.cassandra.transport.Message$ProtocolEncoder.encode(Message.java:392) system.log- at org.apache.cassandra.transport.Message$ProtocolEncoder.encode(Message.java:378) system.log- at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88) system.log- at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) system.log- at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) system.log- at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:816) system.log- at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:723) system.log- at org.apache.cassandra.transport.Message$Dispatcher$ChannelFlusher.add(Message.java:513) system.log- at org.apache.cassandra.transport.Message$Dispatcher$Flusher.run(Message.java:567) system.log- at org.apache.cassandra.concurrent.EpollTPCEventLoopGroup$SingleCoreEventLoop.process(EpollTPCEventLoopGroup.java:924) Can you please help or guide what can be the cause of this issue ?
