question

Naraen avatar image
Naraen asked Erick Ramirez answered

DSBulk reports "Error while opening new channel (ConnectionInitException: [driver|connecting...] Protocol initialization request"

hi,

When I run Dsbulk for data import I see the below mentioned logs. They dont cause any failures in import. Here We use DSBulk to connect with Cassandra clusters running on k8s. Hence Suspecting ingress is loaded and causing these issues?

Username and password provided but auth provider not specified, inferring PlainTextAuthProvider
Operation directory: /opt/virtuoranc/bin/VIRTUORA_BACKUP_RESTORE/log/dsbulkLog/import_20220721-210018/connectedservicestoaid
[driver|/10.36.0.5:9042] Error while opening new channel (ConnectionInitException: [driver|connecting...] Protocol initialization request, step 1 (STARTUP {CQL_VERSION=3.0.0, DRIVER_NAME=DataStax Java driver for Apache Cassandra(R), DRIVER_VERSION=4.7.2, CLIENT_ID=4c533d24-1d9b-3a9b-8232-0fd03d464cab, APPLICATION_NAME=DataStax Bulk Loader connectedservicestoaid, APPLICATION_VERSION=1.7.0}): failed to send request (java.nio.channels.NotYetConnectedException))
java driverdsbulk
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

smadhavan avatar image
smadhavan answered smadhavan commented

@Naraen without the below information it is hard to help triage this. Could you update your original questions and post additional details to better help us help you? Thanks!

  1. DSBulk (./dsbulk --version) [This appears to be 1.7.0 from that error, but good to confirm. ] and the target Cassandra cluster version details. You could download latest DSBulk 1.9.0 from here, https://downloads.datastax.com/#bulk-loader
  2. Full command line options for running DSBulk (of course, with passwords redacted) and the logs from LOAD-datetimestamp directory that gets created when we run the operation
  3. You mentioned "import". There is nothing called import operation in DSBulk and I am assuming that as "dsbulk load" out of the Cassandra cluster
  4. You've mentioned the usage of K8s, are you using K8ssandra clusters on here. What version of K8ssandra and Kubernetes are in use here?
  5. An any other details that you could share here?
  6. How are you claiming that the ingress is affecting here? Any stats/details on that networking that you share here?
2 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Naraen avatar image Naraen commented ·

hi @smadhavan

Thanks for your comments.! I have provided you with some more information here.

Dsbulk Version: 1.7.0

DSbulk Query: ./dsbulk load -url /home/xxx/cassandra_backup/xxx.csv -k dpfmdatastore -t xxx -h <IP> -u cassandra -p cassandra -maxConcurrentQueries 1C --dsbulk.connector.csv.maxCharsPerColumn 4096 -cl LOCAL_QUORUM

I am trying to do Dsbulk Load ie., unloading from a cluster and loading in to another k8s cluster.

I am not using k8ssandra, intstead i use bitnami cassandra image 3.11.8

During loading i am not seeing this issue with cassandra nodes running on VM, Only issue is seen with cassandra running on k8s. That is the reason why i suspect Ingress is causing any issues!

0 Likes 0 ·
smadhavan avatar image smadhavan ♦ Naraen commented ·

sure, thanks. Nothing jumps out to me. I'll see if I can get additional pair of eyes to triage this.

0 Likes 0 ·
Erick Ramirez avatar image
Erick Ramirez answered

The embedded Java driver in the DSBulk utility seems to lose connection to your cluster so it reconnects to the cluster.

However, there was a bug in the older versions of the Java driver where it was scheduling too many reconnections when one has already been established (JAVA-2911). This bug was fixed in v4.10 of the driver.

DSBulk v1.7.0 uses Java driver v4.7.2. DSBulk v1.8.0 uses Java driver v4.10 while the latest DSBulk v1.9.0 has Java driver v4.14.

If you download the latest DSBulk from DataStax Downloads, you should not see multiple instances of the message you reported. Note however that if DSBulk loses connectivity to your cluster, you will still see at least one reconnection message. Cheers!

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.