How to load a bunch of data into LCS level 4

Question

question

sumsionjg_26599 asked Sep 17, '19 pmcfadin answered Sep 18, '19

How to load a bunch of data into LCS level 4

I've got a bulk load process that puts a bunch of data into a table with the STCS compaction (1-2T). Now I want to switch the compaction strategy to LCS, but leave levels L1-L3 empty for future writes (100G). All the bulk load data is read only and won't be updated, so I don't want to pay compaction costs as the relatively small amount of updates come in.

I asked at CassandraSummit and someone pointed me to sstableofflinerelevel, but there is no option in that utility to tell it which level to put the sstables in.

Or does it just push everything up to the highest level?

lcs sstables

2

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

sumsionjg_26599 commented · Sep 17, 2019 at 06:04 PM

this doc didn't tell me anything about which level the tables would end up in: https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/tools/toolsSStables/toolsSStabOfflineRelevel.html

0 ·

sumsionjg_26599 commented · Sep 17, 2019 at 10:17 PM

This is what I got when attempting to run `sstableofflinerelevel ks table`:
```

For sstables in /mnt-data/data/ks/table-be307490f1a311e88f7dbd9b1b842167:

Current leveling:

L0=2

L1=10

L2=113

L3=354

New leveling:

L0=9

L1=17

L2=94

L3=359

```

It didn't hoist things up to L3, which would have fit everything.

0 ·

DataStax Enterprise is powered by the best distribution of Apache Cassandra ™

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Privacy Policy Terms of Use

Answer 1 · 2019-09-18T00:03:16Z

pmcfadin answered Sep 18, '19

The algorithm for LCS is designed to fill each level and then promote. I cover that in a video here: https://academy.datastax.com/units/21015-leveled-compaction-dse-operations-apache?resource=ds210-datastax-enterprise-6-operations-with-apache-cassandra

I don't think what you are asking for fits into how LCS wants to work. Each level is a multiple of max SSTable size so you should have data in each level on a well settled compaction. The 'sstableofflinerelevel' ensures each level is fully populated based on the rules I stated.

If you are looking to avoid compaction, the strategy to use is TWCS, but its only for time series data so may not work for your specific use case.

Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

question