Build Cloud-Native apps with Apache Cassandra

GOT QUESTIONS from the Workshop? You're in the right place! Post a question here and we'll get you answers.

Click here for Week 7 Materials and Homework.

Registrations still open!


question

Tri avatar image
Tri asked ·

What is a reasonable size limit of a CSV for the COPY FROM command in cqlsh?

DS210 CQL COPY Course: https://academy.datastax.com/units/21022-cql-copy-dse-operations-apache-cassandra

OK for using COPY in production DB for small amount of data

Can you define "small amount" ?

Or more exactly, what is the practicable size limit of a CSV (in rows, columns, CSV file size?) before the use of COPY FROM is discouraged? A ball park number would be enough.

ds210cqlshcopy
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

saravanan.chinnachamy_185977 avatar image
saravanan.chinnachamy_185977 answered ·

@Tri The COPY command can be used efficiently for copying few thousand records with out running into major problems.

For large datasets running into millions, it is advised to use DataStax Bulk Loader.

DataStax Bulk Loader efficiently and reliably loads small or large amounts of data. CSV or JSON files can be rapidly loaded or unloaded to or from the following supported databases:

For an in-depth details on these tools, please listen to a video from DataStax team at

https://www.youtube.com/watch?v=CAH7Mlg_rVI

Also please refer to our Blog post on Bulk Loader

https://www.datastax.com/blog/2018/05/introducing-datastax-bulk-loader

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.