Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

gmldba_107428 avatar image
gmldba_107428 asked ·

Should I worry when I see sstables increasing when I am adding more data than before?

My writes per second had double from 750 writes per second to about 1500 writes per second in the last 7 days. sstable counts keeps increasing even though we have been an avarage of 1500 writes per second for a while. What else can I lok for?

performance
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image
Erick Ramirez answered ·

The quick answer is no. When you're inserting lots of data, it follows that you should see more data files.

I know you did the certification so you should know that SSTables are immutable -- they don't change once they've been flushed to disk or written out by compaction.

New mutations will get written to new SSTables. It's just a normal part of Cassandra's operation. Cheers!

1 comment Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thank you. I was double checking in case I missed something.

0 Likes 0 · ·
bettina.swynnerton avatar image
bettina.swynnerton answered ·

Hi @gmldba_107428,

without more details about the quantitative trend that you are seeing, it's difficult to comment whether this is reason to worry.

If you add more data, more data is stored in sstables, and yes, an increase is expected. The question is whether the increase is out of proportion. How many sstables per table are we talking about, and how much data is in the respective tables?

One reason for a significantly increasing sstable count is when compactions cannot keep up. You can check on pending compactions with nodetool compactionstats. If you see a lot of pending compactions, it would perhaps make sense to increase the compaction throughput and monitor if this brings the pending compactions back to 0 or 1.

Here for documentation:

Monitoring compaction statistics:

https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/tools/toolsCompactionStats.html

Getting and setting compaction throughput:

https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/tools/toolsGetcompactionthroughput.html

https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/tools/toolsSetCompactionThroughput.html

Hope this gives you a bit more insight what is going on.

1 comment Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thank you. Just double checking. there are no pending compactions.

0 Likes 0 · ·