Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

kinomoto avatar image
kinomoto asked ·

Stream failed due to CompactionExecutor

While joining a node to the cluster, we intermittently get:

ERROR [CompactionExecutor:2] 2021-02-03 12:06:09,966 CassandraDaemon.java:228 - Exception in thread Thread[CompactionExecutor:2,1,main]java.util.NoSuchElementException: null

This is immediately followed by:

ERROR [StreamReceiveTask:19] 2021-02-03 12:06:09,966 StreamSession.java:593 - [Stream #fba1aa60-6613-11eb-974b-391d84d34cb8] Streaming error occurred on session with peer 10.51.41.58
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.util.NoSuchElementException

Then it sends a fail message to the peer:

DEBUG [STREAM-OUT-/10.51.41.58:7001] 2021-02-03 12:06:09,967 ConnectionHandler.java:379 - [Stream #fba1aa60-6613-11eb-974b-391d84d34cb8] Sending Session Failed

On the peer's log there's only:

ERROR [STREAM-IN-/10.51.41.79:34226] 2021-02-03 12:06:09,970 StreamSession.java:706 - [Stream #fba1aa60-6613-11eb-974b-391d84d34cb8] Remote peer 10.51.41.79 failed stream session.

Any tips how to troubleshoot these and get rid of them? The bootstrap eventually fails and we believe it is because of these errors. Thanks!

compaction
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

You will need to review the logs on the source node to determine why the compaction thread CompactionExecutor:2 is getting a NoSuchElementException.

You will need to pay particular attention to the SSTable(s) which are generating the exception. Look into the schema + contents of the problematic SSTable(s) for clues as to why the compaction thread is failing. Cheers!

2 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks Erick. It turned out to be a stack of problems. Data, JVM, settings related. So, there is no simple solution. Working with Datastax support now to get things sorted out.

0 Likes 0 ·

Good to hear that you're working with DataStax Support. You're in good hands. Cheers!

0 Likes 0 ·