Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

ritu.kumari1_180415 avatar image
ritu.kumari1_180415 asked ·

How do we track total records inserted in a table in a day?

Hi Folks,

There is a requirement where we need to track total records inserted in table in a day.What is the best way to do this.

Thanks

Ritu

cassandra
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

There isn't an out-of-the-box solution for this kind of use case. Typically you will need write an app that would do this for you or run a Spark job.

I don't recommend using the CQL COUNT() function because it will affect the performance of your cluster. I've discussed it in detail in this post -- Why COUNT() is bad in Cassandra.

Alternatively, you can use the COUNT command in the DataStax Bulk Loader (DSBulk) utility. At the same time each, you can run a dsbulk count on a table to get the total records and subtract the previous day's total to get today's tally.

The challenge with counting records is figuring out how to deal with updates and deletes. Unless you're just inserting new records every day, it's almost impossible to reconcile how many are new and how many were deleted. Cheers!

3 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks @Erick Ramirez. Actually requirement is to get daily records inserted .One way is to use table cfhistogram and other is get partitions details and query for starting and ending range using select. Is there any other way which is more efficient.

0 Likes 0 ·

Like I said, you can use DSBulk to do the counts and you can use the methodology I outlined. Cheers!

0 Likes 0 ·

Hi erick, thanks for share the solution

0 Likes 0 ·