I have to delete 3 million records to free up space .How should i proceed .Should i do a batch of 10000 records first and weight for the gc_grace period to finish so that it will clean up the tombstone or run run manual compaction to remove the tombstones immediately .
Bringing together the Apache Cassandra experts from the community and DataStax.
Want to learn? Have a question? Want to share your expertise? You are in the right place!
Not sure where to begin? Getting Started
This question was
closed
by
Erick Ramirez for the following reason: The question is answered, right answer was accepted
How should I proceed deleting 3 million records?
There's no ideal way to handle mass deletions in Cassandra.
If you're deleting a range of rows within partitions, this is going to be problematic since reading partitions will require iterating over the deleted rows. But if you are deleting whole partitions, it's not a real concern since it won't suffer from the same problem as deleted rows.
Our general recommendation is to stretch out the deletions so the tombstones don't put pressure on your cluster. Break the deletions up over a long period of time as much as possible.
Be aware that running manual compactions has its own problems and is only relevant for compaction strategies like SizeTieredCompactionStrategy. I've written about this problem previously in Why forcing a major compaction is not ideal.
It's also important to note that mass deletions on tables using LeveledCompactionStrategy can result in an IO storm since LCS will aggressively merge partition fragments together into SSTables. Cheers!
Thanks for the help @Erick Ramirez.I will stretch out the deletions by deleting them in small batches .
5 People are following this question.
