question

jtdelato avatar image
jtdelato asked Erick Ramirez commented

Problem loading CSV to Astra using DSBulk

I have tried to find the answer to my problem among other forum posts and in the Bulk Loader and Astra documentation, but have been unsuccessful. So, I apologize if this has been answered somewhere else.

I am trying to practice loading a CSV file to Astra using Bulk Loader on my Windows laptop. I am using the following command under my \dsbulk-1.8-0\bin directory:

\dsbulk-1.8.0\bin>dsbulk load -url test.csv -k mykeyspace -t mytable \
 -b "secure-bundle-path.zip" -u user -p password -header true

The operation appears to start and I see the following:

Username and password provided but auth provider not specified, inferring PlainTextAuthProvider
A cloud secure connect bundle was provided: ignoring all explicit contact points.
A cloud secure connect bundle was provided and selected operation performs writes: changing default consistency level to LOCAL_QUORUM.
Operation directory: C:\Users\JT\Desktop\dsbulk-1.8.0\bin\logs\LOAD_20210406-172216-572000

However, the output stops there.

I understand that with a free Astra account, the load time may be slower, but I have let it run for 30 minutes or more, and have tried several times, with no success. It never produces the expected output showing rows being processed.

I do not believe it is my password or authentication as that error appears immediately if I have entered my user/password incorrectly in the command (side note: my user and password work, but my App token is always rejected for some reason - my authentication configuration I'm guessing?)

Does anyone have any ideas or suggestions as to what may be happening?

I have the latest JDK installed and have set up both the JAVA_HOME and Path variable.

Do I need to do anything else with my environment variables, like adding Bulk Loader to my Path variable?

Are there any minimum hardware requirements for Bulk Loader that I may not be meeting with my basic laptop?

Thanks so much for any ideas or help!

astradsbulk
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

You didn't provide any errors in your post so it's hard for us to determine the problem.

However, I suspect it's because you're using the wrong credentials. Instead of username and password, you should be using the client ID + secret.

I've responded to a similar issue yesterday where I provided a full working example for loading data to Astra using DSBulk. The command should be:

$ dsbulk load -url /path/to/input.csv -header true -k ks_name -t table_name -b "/path/to/secure-connect-db.zip" -u client_id -p client_secret

For full details of the working example, see question #10901. Cheers!

6 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

jtdelato avatar image jtdelato commented ·

Hi Erick,

Thanks!

I have actually already consulted your wonderfully detailed response to question #10901.

I receive no error message when using my regular user/password. The operation just simply stops and doesn't continue with output past:

Username and password provided but auth provider not specified, inferring PlainTextAuthProvider
A cloud secure connect bundle was provided: ignoring all explicit contact points.
A cloud secure connect bundle was provided and selected operation performs writes: changing default consistency level to LOCAL_QUORUM.
Operation directory: C:\Users\JT\Desktop\dsbulk-1.8.0\bin\logs\LOAD_20210407-005122-037000

When I have tried to use my App Token credentials, I receive an error message that I have attached here as a screenshot.

Do I need to change any authentication configuration for my Astra instance in order for my token to work?

Thanks again!

Error Message Screenshot.png

0 Likes 0 ·
error-message-screenshot.png (153.5 KiB)
Erick Ramirez avatar image Erick Ramirez ♦♦ jtdelato commented ·

It sounds like you have a "classic DB" instance. As I stated in #10901, IAM was implemented recently in Astra (early March) so if you created your DB before that time then you have a classic instance.

Have you checked the DSBulk logs for clues as to where it's possibly stuck?

0 Likes 0 ·
jtdelato avatar image jtdelato Erick Ramirez ♦♦ commented ·

Thanks!

I definitely have a classic DB.

My logs only show the same output that was displayed in my console, with the addition of two lines from when I manually terminated the operation 30 minutes later.

I have attached a screenshot of my log file.

If it means anything, I am having the exact same issue if I try to run a Count operation with DSBulk.

Thanks so much for all your help!DSBulk Log Screenshot.png

0 Likes 0 ·
dsbulk-log-screenshot.png (108.2 KiB)
Show more comments