Hi Folks,
There is a requirement where we need to track total records inserted in table in a day.What is the best way to do this.
Thanks
Ritu
There isn't an out-of-the-box solution for this kind of use case. Typically you will need write an app that would do this for you or run a Spark job.
I don't recommend using the CQL COUNT() function because it will affect the performance of your cluster. I've discussed it in detail in this post -- Why COUNT() is bad in Cassandra.
Alternatively, you can use the COUNT command in the DataStax Bulk Loader (DSBulk) utility. At the same time each, you can run a dsbulk count on a table to get the total records and subtract the previous day's total to get today's tally.
The challenge with counting records is figuring out how to deal with updates and deletes. Unless you're just inserting new records every day, it's almost impossible to reconcile how many are new and how many were deleted. Cheers!
Thanks @Erick Ramirez. Actually requirement is to get daily records inserted .One way is to use table cfhistogram and other is get partitions details and query for starting and ending range using select. Is there any other way which is more efficient.
Like I said, you can use DSBulk to do the counts and you can use the methodology I outlined. Cheers!
7 People are following this question.
