question

tyuen_144153 avatar image
tyuen_144153 asked Erick Ramirez answered

Cannot load data for a table with CQL type duration with C* 3.11.5 sstableloader

I have a Cassandra table with the data type "duration". However, when I tried to use to sstableloader to restore the snapshot into another cluster with the duration data type, I'm getting cannot resolve user type.

The duration data type is a native type within Cassandra. I'm running Cassandra 3.11.3 community edition.

ERROR 17:38:10,726 Error parsing schema for table datakeyspace.retrylog: Cluster.getMetadata().getKeyspace("datakeyspace").getTable("retrylog") will be missing or incomplete
com.datastax.driver.core.exceptions.UnresolvedUserTypeException: Cannot resolve user type datakeyspace.duration
        at com.datastax.driver.core.DataTypeCqlNameParser.parse(DataTypeCqlNameParser.java:147) ~[cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.TableMetadata.build(TableMetadata.java:188) ~[cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.SchemaParser.buildTables(SchemaParser.java:176) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.SchemaParser.buildKeyspaces(SchemaParser.java:128) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.SchemaParser.refresh(SchemaParser.java:64) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.ControlConnection.refreshSchema(ControlConnection.java:343) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:273) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:201) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:79) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1424) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.Cluster.init(Cluster.java:163) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:334) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:309) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at com.datastax.driver.core.Cluster.connect(Cluster.java:251) [cassandra-driver-core-3.0.1-shaded.jar:na]
        at org.apache.cassandra.utils.NativeSSTableLoaderClient.init(NativeSSTableLoaderClient.java:73) [apache-cassandra-3.11.3.jar:3.11.3]
cassandrasstableloader
2 comments
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

laxmikant.hcl_32751 avatar image laxmikant.hcl_32751 commented ·

is your issue resolved? Have you tried with cassandra-driver-core-3.3.1.jar

0 Likes 0 ·
tyuen_144153 avatar image tyuen_144153 laxmikant.hcl_32751 commented ·

this is still an issue. it has not resolved.

0 Likes 0 ·

2 Answers

Erick Ramirez avatar image
Erick Ramirez answered

Apologies that your question got buried in the backlog. I've finally managed to circle back and I've been working on a possible fix for you. Let me document the details here for posterity.

Symptom

Tools in Cassandra 3.x cannot parse the schema of tables which have columns of CQL type duration.

Cause

The CQL duration type was added in Cassandra 3.10 (CASSANDRA-11873) to support queries with a filter based on a time interval, for example:

SELECT ... FROM ... WHERE pk = ? AND reading_time < now() - 2h

Support for the duration type was added to the drivers including the Java driver 3.2.0 (JAVA-1347). However, Cassandra 3.11 ships with an older shaded version of the Java driver which doesn't have support for the duration type:

lib/cassandra-driver-core-3.0.1-shaded.jar

Workaround

Please test this procedure and let me know how it goes.

Step 1 - Download a newer version of the Java driver from Maven in the same 3.x family.

For example, download cassandra-driver-core-3.10.0-shaded.jar from cassandra-driver-core 3.10.0 on Maven.

Step 2 - On the sstableloader node, replace the old JAR with cassandra-driver-core-3.10.0-shaded.jar.

Step 3 - Attempt to load the data again with sstableloader.

Solution

I have logged CASSANDRA-16005 and recommended we update the Java driver that is shipped with Cassandra 3.11.

Note the issue does not apply to Cassandra 4.0+ since it ships with a version of the Java driver that includes JAVA-1347. Cheers!

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

laxmikant.hcl_32751 avatar image
laxmikant.hcl_32751 answered tyuen_144153 commented

Regarding duration datatype error , you need to use cassandra-driver-core version is 3.2.0 or higher ( Ref: https://datastax-oss.atlassian.net/browse/JAVA-1347)

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

tyuen_144153 avatar image tyuen_144153 commented ·

hi, do you any info on how to upgrade the Cassandra's cassandra-driver-core-3.0.1-shaded.jar to be 3.2.0 or higher? The Cassandra version is 3.11.5

0 Likes 0 ·