question

LifeIsGoodMF avatar image
LifeIsGoodMF asked Erick Ramirez commented

Why do older SSTables exist for a table with TWCS and 30-day TTL?

Hello smart guys,
have a table with the following settings:

and compaction = {
  'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 'compaction_window_size': '1', 'compaction_window_unit': 'DAYS', 'max_threshold': '32', 'min_threshold': '4'}
and default_time_to_live = 2629800
and gc_grace_seconds = 86400;

that suppose to have only the last 30 days of data but i see SSTables older than that.

Could someone tell me why GC is not happening?

twcsttl
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

robbyb_174297 avatar image
robbyb_174297 answered

The tables are retained for the TTL + gc_grace_seconds + a bit of spare buffering. Why would you have gc_grace_seconds set to anything other than 0 on a twcs table?

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

It is likely that data was inserted and it hasn't expired yet so it is blocking some SSTables from getting dropped.

Try running the sstableexpiredblockers utility to get a list of blocking SSTables then dump the contents of the offending files to see the timestamp range of data they contain. Cheers!

2 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

LifeIsGoodMF avatar image LifeIsGoodMF commented ·

Thanks. Is it optimized somehow or goes through all the values? Just trying to estimate the maintenance window.

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ LifeIsGoodMF commented ·

It checks each SSTables' metadata to figure out which expired SSTables are blocked by another SSTable. To be clear, it compares the metadata of the SSTables, not the data in them. Cheers!

1 Like 1 ·