I have a table where nodesync is enabled. When I run a DSBULK on that table, I see compactions triggering continuously which I think is delaying the load. Is it a good practice to disable nodesync while loading tables with DSBULK.
Hi! Nodesync works on the write path as opposed to the repair service which uses the streaming path. With that said, if you are inserting a lot of data it might help to temporarily disable nodesync on the table while you load the data to prevent write amplification.
DSBulk doesn't really have anything to do with NodeSync or compactions. I think you're just conflating them unnecessarily.
I've explained in your other question (#13285) that compactions are part of the normal operation of Cassandra. They get triggered whether you are bulk-loading data or not.
Similarly, NodeSync (and repairs for that matter) are part of keeping data consistent across nodes in your cluster and is necessary given the distributed architecture of Cassandra. It is normal for repairs and NodeSync to be running in the background.
The fact that you think there's a delay while you are loading data just indicates that you haven't sized your cluster correctly. Remember that Cassandra scales linearly --
- IF your 3-node cluster than can sustain 200K ops/sec
- AND you need a throughput of 400K ops/sec
- THEN double your cluster to 6 nodes to achieve double the throughput.
Based on your previous questions, I think we've established that your cluster does not have sufficient capacity to sustain the throughput you require so you should resize your cluster accordingly, even if it's just a temporary measure while you are migrating data. There's no cost associated with adding nodes temporarily since DSE subscriptions allow for temporary "bursting" during peak events like Black Friday sales. Cheers!
7 People are following this question.
