Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

Tonny avatar image
Tonny asked Erick Ramirez edited

cassandra 在读请求较大的情况下 jvm STW 时间过长,频繁出现concurrent mode failure

cassandra采用默认配置,在4U16G规格机器上,参数配置如下:

Xms=4G,Xmx=4G,Xmn=400M,-XX:SurvivorRatio=8,-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75

较频繁出现STW持续时间达7s

2020-11-09T11:58:56.491+0000: 30505.639: Total time for which application threads were stopped: 0.0250854 seconds, Stopping threads took: 0.0000859 seconds
{Heap before GC invocations=8849 (full 140):
 par new generation   total 368640K, used 367676K [0x00000006c0000000, 0x00000006d9000000, 0x00000006d9000000)
  eden space 327680K, 100% used [0x00000006c0000000, 0x00000006d4000000, 0x00000006d4000000)
  from space 40960K,  97% used [0x00000006d6800000, 0x00000006d8f0f3a0, 0x00000006d9000000)
  to   space 40960K,   0% used [0x00000006d4000000, 0x00000006d4000000, 0x00000006d6800000)
 concurrent mark-sweep generation total 3784704K, used 3717119K [0x00000006d9000000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 43699K, capacity 45400K, committed 48724K, reserved 1091584K
  class space    used 4689K, capacity 5045K, committed 6032K, reserved 1048576K
2020-11-09T11:58:56.567+0000: 30505.715: [GC (Allocation Failure) 2020-11-09T11:58:56.568+0000: 30505.715: [ParNew: 367676K->367676K(368640K), 0.0000289 secs]2020-11-09T11:58:56.568+0000: 30505.715: [CMS2020-11-09T11:58:57.556+0000: 30506.703: [CMS-concurrent-mark: 1.147/1.273 secs] [Times: user=5.09 sys=0.06, real=1.28 secs] 
 (concurrent mode failure): 3717119K->3574572K(3784704K), 7.1465703 secs] 4084796K->3574572K(4153344K), [Metaspace: 43699K->43699K(1091584K)], 7.1468865 secs] [Times: user=12.21 sys=0.06, real=7.15 secs] 
Heap after GC invocations=8850 (full 141):
 par new generation   total 368640K, used 0K [0x00000006c0000000, 0x00000006d9000000, 0x00000006d9000000)
  eden space 327680K,   0% used [0x00000006c0000000, 0x00000006c0000000, 0x00000006d4000000)
  from space 40960K,   0% used [0x00000006d6800000, 0x00000006d6800000, 0x00000006d9000000)
  to   space 40960K,   0% used [0x00000006d4000000, 0x00000006d4000000, 0x00000006d6800000)
 concurrent mark-sweep generation total 3784704K, used 3574572K [0x00000006d9000000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 43699K, capacity 45400K, committed 48724K, reserved 1091584K
  class space    used 4689K, capacity 5045K, committed 6032K, reserved 1048576K
}
2020-11-09T11:59:03.714+0000: 30512.862: Total time for which application threads were stopped: 7.1505311 seconds, Stopping threads took: 0.0022926 seconds

有什么比较有效的调优办法,降低STW的时长

jvm
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

4GB对于Cassandra工作负载而言太小了。对于16GB的RAM,我建议将heap max size增大到8GB,将minimum size增大到800MB。

让我们看看它在此配置下的性能。

4GB is too small for C workloads. With 16GB of RAM, I recommend increasing the heap max size to 8GB and minimum size to 800MB.

Let's see how it performs with this configuration. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.