PLANNED MAINTENANCE

Hello, DataStax Community!

We want to make you aware of a few operational updates which will be carried out on the site. We are working hard to streamline the login process to integrate with other DataStax resources. As such, you will soon be prompted to update your password. Please note that your username will remain the same.

As we work to improve your user experience, please be aware that login to the DataStax Community will be unavailable for a few hours on:

  • Wednesday, July 15 16:00 PDT | 19:00 EDT | 20:00 BRT
  • Thursday, July 16 00:00 BST | 01:00 CEST | 04:30 IST | 07:00 CST | 09:00 AEST

For more info, check out the FAQ page. Thank you for being a valued member of our community.


question

satvantsingh_190085 avatar image
satvantsingh_190085 asked ·

What is the purpose of memtable_heap_space_in_mb and memtable_offheap_space_in_mb?

Hi Erick,

What is the purpose of "off heap size" & " memtable off heap size" in cassandra.yaml file. Also what is the recommended settings for "off heap" sizes ?


cassandraperformanceconfigurationmemtables
1 comment
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

@satvantsingh_190085 update your post to provide the version of OSS C* or DSE that you're using on here for added clarity

0 Likes 0 · ·
bettina.swynnerton avatar image
bettina.swynnerton answered ·

Hi,

it is not quite clear from your questions, so I assume you refer to the settings memtable_heap_space_in_mb and memtable_offheap_space_in_mb in the cassandra.yaml ( I assume that you are referring to OSS C* 3.11).

The yaml has two settings for memtable sizes, on-heap and off-heap heap. They both default to 1/4 of the total heap size, and this default is probably a good starting point until you are certain how your workload behaves and want to consider more detailed tuning. If you want to make changes, proceed with small changes and monitor carefully, as allocating memory for memtables reduces the overall amount of memory that Cassandra can use.

Cassandra stores memtables either on the Java heap, off-heap (native) memory, or both. The memtable allocation is the total upper limit for memory that Cassandra can use for memtables. As per the yaml comments, Cassandra will stop accepting writes when the limit is exceeded until a flush completes. A flush is triggered based on the memtable_cleanup_threshold.

By default, the memtable_allocation_type is set to heap_buffers, which allocates memtables on the heap using Java NIO. If necessary and to alleviate heap pressure, you can change the allocation type to off-heap. These are tuning changes and should be done within the context of your workload performance.

I hope this helps.

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

smadhavan avatar image
smadhavan answered ·

@satvantsingh_190085, I am making an assumption in here that you're using DSE 6.8 and with that memtable_offheap_space_in_mb property is deprecated and instead you're supposed to be using memtable_space_in_mb property which is defaulted to 2048. Description for this property is as follows,

Total permitted memory to use for memtables. When this threshold is exceeded, writes are not accepted until a flush completes. A flush is triggered based on memtable_cleanup_threshold. When not set:

If the deprecated settings are not present, the calculated default is 1/4 of the heap size.

See Tuning Java Virtual Memory for additional information!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.