question

adityajain22_141041 avatar image
adityajain22_141041 asked Erick Ramirez commented

High GC pause on one particular node

Hi @Erick Ramirez,


Out of 18 node cluster, we are facing slow queries alert and high GC pause in One Particular node.


Below Hyperlink is for Gc analysis report, can you go through and check

https://gceasy.io/diamondgc-report.jsp?p=c2hhcmVkLzIwMjAvMDMvMjAvLS1nYy5sb2cuNS5jdXJyZW50LnppcC0tMTItMTItNDA=&channel=WEB



gc
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

@adityajain22_141041 There isn't enough information in your post to figure out possible causes of the issue. My best guess is that it's highly likely that your application is not configured correctly and is sending all traffic to just one node in your cluster.

You can verify my hypothesis by running netstat on the node and looking at all the connections to the CQL port (default 9042) and compare it to the other nodes. You want to pay particular attention to the source IP or hostname since it might give you clues (for example a rogue app server). Cheers!

5 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

adityajain22_141041 avatar image adityajain22_141041 commented ·

Hi @Erick

We checked load distribution is in order among the cluster,

Can you review our gc configs mentioned in above report.


That would be great help.

Thanks


0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ adityajain22_141041 commented ·

The link you posted isn't accessible. Plus it isn't something we do. I'm happy to provide you pointers in terms of possible causes but ultimately, you'll need to do the investigation yourself. Cheers!

0 Likes 0 ·
adityajain22_141041 avatar image adityajain22_141041 Erick Ramirez ♦♦ commented ·

Working link gor GC report

https://gceasy.io/my-gc-report.jsp?p=c2hhcmVkLzIwMjAvMDMvMjQvLS1nYy5sb2cuNS5jdXJyZW50LnppcC0tMTQtMy00Nw==&channel=WEB

0 Likes 0 ·
Show more comments
adityajain22_141041 avatar image adityajain22_141041 commented ·

Hi @Erick

We checked load distribution is in order among the cluster,

Can you review our gc configs mentioned in above report.


That would be great help.

Thanks

0 Likes 0 ·