PLANNED MAINTENANCE

Hello, DataStax Community!

We want to make you aware of a few operational updates which will be carried out on the site. We are working hard to streamline the login process to integrate with other DataStax resources. As such, you will soon be prompted to update your password. Please note that your username will remain the same.

As we work to improve your user experience, please be aware that login to the DataStax Community will be unavailable for a few hours on:

  • Wednesday, July 15 16:00 PDT | 19:00 EDT | 20:00 BRT
  • Thursday, July 16 00:00 BST | 01:00 CEST | 04:30 IST | 07:00 CST | 09:00 AEST

For more info, check out the FAQ page. Thank you for being a valued member of our community.


question

arsen.gasparyan.05_188257 avatar image
arsen.gasparyan.05_188257 asked ·

Can't do update query via Kafka Connector.

I have the following Table

CREATE TABLE test.test ( id int, change_log list<text>, PRIMARY KEY (id) ) ;

Value : {"changeLog":["text"],"id":1234}

When I want to run update query by Kafka connector

topic.Test.test.test.query=UPDATE test.test SET change_log = change_log +  :changeLog WHERE id = :id ;

I get the following issue

ERROR Problem when getting queryFuture. This is likely a bug in the connector, please report. (com.datastax.kafkaconnector.DseSinkTask:153)
java.util.concurrent.ExecutionException: com.datastax.oss.driver.api.core.servererrors.ServerError: java.lang.ClassCastException: org.apache.cassandra.cql3.Constants$Value cannot be cast to org.apache.cassandra.cql3.Lists$Value
cassandraconnectorcqlcollectionskafka
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Tomasz Lelek avatar image
Tomasz Lelek answered ·

Please take a look at the logs that are produced by the Kafka connector.

When the prepared statement is produced you should be able to see the log starting with:

[2020-06-10 10:19:13,879] ERROR WorkerSinkTask{id=dse-sink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
java.lang.RuntimeException: Prepare failed for statement: UPDATE my_ks.test SET change_log = change_log + [:changeLog] WHERE id = :id ;

When you will scroll down at the end of the stack trace of this log you will see the underlying cause of your problems:

Caused by: com.datastax.oss.driver.api.core.servererrors.InvalidQueryException: Invalid list literal for change_log: bind variables are not supported inside collection literals

The error denotes that you cannot use bind variables (:changeLog) for collections.

It’s a limitation of the CQL itself.

I am enclosing the full exception log for a reference: Invalid_list_literal_error.txt


Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

alex.ott avatar image
alex.ott answered ·

Unfortunately, CQL doesn't support binding inside the collection. If you do similar query inside the Java driver, you can get more detailed error, like:

com.datastax.driver.core.exceptions.InvalidQueryException: Invalid set literal for l: bind variables are not supported inside collection literals

To perform what you need to you need to modify your topic values to wrap that :changeLog value into a list instead of the single value, and then use query like this:

topic.Test.test.test.query=UPDATE test.test SET change_log = change_log + :changeLog WHERE id = :id ;

Transformation of single value into a list/array theoretically could be done with KSQL, or maybe custom transformations inside Kafka Connect framework.

P.S. I answered similar question on StackOverflow yesterday.

2 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Anyway with list also I am reseving the same error

{"changeLog":["Hkf"],"id":178}

topic.Test.test.test.query=UPDATE test.test SET change_log = change_log + :changeLog WHERE id = :id ;

This is likely a bug in the connector, please report. (com.datastax.kafkaconnector.DseSinkTask:153)

java.util.concurrent.ExecutionException: com.datastax.oss.driver.api.core.servererrors.ServerError: java.lang.ClassCastException: org.apache.cassandra.cql3.Constants$Value cannot be cast to org.apache.cassandra.cql3.Lists$Value


0 Likes 0 · ·
alex.ott avatar image alex.ott arsen.gasparyan.05_188257 ·

You don't have list in your Kafka topic - you have single value that you need to transform to list using one of the methods I described (custom transformations, or KSQL).

0 Likes 0 · ·
smadhavan avatar image
smadhavan answered ·

@arsen.gasparyan.05_188257, it appears that the table schema expect change_log column value to be a list<text>`` whereas the actual JSON structure has it as text type: {..."changeLog":"text"...}. How about if you fix the JSON to have an actual list of strings something like {..."changeLog":["text"]...}?

Couple other references,

2 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

I also try as you mentioned and I have the following

ERROR Problem when getting queryFuture. This is likely a bug in the connector, please report. (com.datastax.kafkaconnector.DseSinkTask:153)

java.util.concurrent.ExecutionException: com.datastax.oss.driver.api.core.servererrors.ServerError: java.lang.ClassCastException: org.apache.cassandra.cql3.Constants$Value cannot be cast to org.apache.cassandra.cql3.Lists$Value


0 Likes 0 · ·
alexandre.dutra avatar image alexandre.dutra ♦ arsen.gasparyan.05_188257 ·

This error seems strange, can you explain a bit more? It seems that the query you provided does not match the data you sent.

0 Likes 0 · ·