If we defined the tables based on the queries we have at the time, and at a later point in time, a new requirement comes along such as a new web page to show the data in a different way, am I understanding it correctly that the application needs to be modified to write data to this new table and the old data would have to be backfilled on to this new table so that the query can work on the old data? How would such a backfill be implemented?
Typically, you'd write an app to read the data from the existing table, do the necessary transformations then write the data to the new table. For large tables, the most efficient way of migrating the data is with Spark.
But the easiest way of doing it is with the DataStax Bulk Loader (DSBulk). First you would unload the data from the existing table to a CSV file then re-organise it so the columns match the schema of the new table then load the modified CSV to the new table. Cheers!
6 People are following this question.
