Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

adityaparikh avatar image
adityaparikh asked Erick Ramirez answered

How do I backfill data from old table to new table?

If we defined the tables based on the queries we have at the time, and at a later point in time, a new requirement comes along such as a new web page to show the data in a different way, am I understanding it correctly that the application needs to be modified to write data to this new table and the old data would have to be backfilled on to this new table so that the query can work on the old data? How would such a backfill be implemented?

migration
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered

Typically, you'd write an app to read the data from the existing table, do the necessary transformations then write the data to the new table. For large tables, the most efficient way of migrating the data is with Spark.

But the easiest way of doing it is with the DataStax Bulk Loader (DSBulk). First you would unload the data from the existing table to a CSV file then re-organise it so the columns match the schema of the new table then load the modified CSV to the new table. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.