Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

sumsionjg_26599 avatar image
sumsionjg_26599 asked pmcfadin answered

How to load a bunch of data into LCS level 4

I've got a bulk load process that puts a bunch of data into a table with the STCS compaction (1-2T). Now I want to switch the compaction strategy to LCS, but leave levels L1-L3 empty for future writes (100G). All the bulk load data is read only and won't be updated, so I don't want to pay compaction costs as the relatively small amount of updates come in.

I asked at CassandraSummit and someone pointed me to sstableofflinerelevel, but there is no option in that utility to tell it which level to put the sstables in.

Or does it just push everything up to the highest level?

lcssstables
2 comments
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

sumsionjg_26599 avatar image sumsionjg_26599 ·

this doc didn't tell me anything about which level the tables would end up in: https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/tools/toolsSStables/toolsSStabOfflineRelevel.html

0 Likes 0 ·
sumsionjg_26599 avatar image sumsionjg_26599 ·

This is what I got when attempting to run `sstableofflinerelevel ks table`:
```

For sstables in /mnt-data/data/ks/table-be307490f1a311e88f7dbd9b1b842167:

Current leveling:

L0=2

L1=10

L2=113

L3=354

New leveling:

L0=9

L1=17

L2=94

L3=359

```

It didn't hoist things up to L3, which would have fit everything.

0 Likes 0 ·

1 Answer

pmcfadin avatar image
pmcfadin answered

The algorithm for LCS is designed to fill each level and then promote. I cover that in a video here: https://academy.datastax.com/units/21015-leveled-compaction-dse-operations-apache?resource=ds210-datastax-enterprise-6-operations-with-apache-cassandra

I don't think what you are asking for fits into how LCS wants to work. Each level is a multiple of max SSTable size so you should have data in each level on a well settled compaction. The 'sstableofflinerelevel' ensures each level is fully populated based on the rules I stated.


If you are looking to avoid compaction, the strategy to use is TWCS, but its only for time series data so may not work for your specific use case.


Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.