question

sebinn avatar image
sebinn asked Erick Ramirez answered

Got cassandra timeout error

---This is the error i got.because of not getting response from cassandra pod my application pod got restarted.I couldn't find any error logs from that application pod.

INFO [SharedPool-Worker-137] 2022-01-31 20:18:30,020 Message.java:609 - Unexpected exception during request; channel = [id: 0x3a2a0bf5, L:/192.168.148.12:9042 ! R:/192.168.148.44:33936]

io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed: Connection reset by peer

at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]

INFO [SharedPool-Worker-34] 2022-01-31 20:18:30,020 Message.java:609 - Unexpected exception during request; channel = [id: 0x8d014349, L:/192.168.148.12:9042 ! R:/192.168.174.183:35744]

io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed: Connection reset by peer

at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]

INFO [Service Thread] 2022-01-31 20:18:32,465 GCInspector.java:284 - ParNew GC in 237ms. CMS Old Gen: 1140193672 -> 1198931112; Par Eden Space: 167772160 -> 0;

cassandra
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

The log entries you posted aren't actually errors. They just indicate that the node (192.168.148.12) tried to respond back to clients but the connection was already closed because the clients gave up.

They're just informational log entries which is why they are logged at INFO level instead of WARN or ERROR.

The client/driver gives up on a request if the cluster doesn't respond within a [configurable] client timeout period. The most common cause of this is that nodes are unresponsive so the coordinator doesn't respond to the client/driver in time.

Nodes are normally unresponsive when they're overloaded. The usual symptoms include lots of GC pauses (or 1 long pause), and/or the disks not able to keep up with the IO requests. Cheers!

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.