Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

rajivbandi_122971 avatar image
rajivbandi_122971 asked jaroslaw.grabowski_50515 commented

Will spark-cassandra-connector 2.5 be updated to include Java driver 4.9?

Currently spark-cassandra-connector 2.5.1 supports cassandra OSS driver 4.7.2

We use this version of the connector as we use Spark 2.4.

Currently Cassandra OSS driver has got upgraded to 4.9.0.

So, will spark-cassandra-connector 2.5.x version be upgraded to include version 4.9.0 or above ?

If not, can I exclude the cassandra driver in the connector and include the latest version of cassandra driver ?

spark-cassandra-connector
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

jaroslaw.grabowski_50515 avatar image
jaroslaw.grabowski_50515 answered jaroslaw.grabowski_50515 commented

Hi! The update should be straightforward and we could definitely do that before the next release. I created a task: https://datastax-oss.atlassian.net/browse/SPARKC-637.

In the meantime you could try the exclusion that you described. Imho it should work but I can't give you a 100% guarantee.

2 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thank you for creating the task, Jaroslaw. I see that you have closed it now. When can I expect the 2.5.2 version to be released ?

0 Likes 0 ·

Not quite sure yet. I hope it's going to happen soon but we still have 3 unresolved 2.5.2 issues.

0 Likes 0 ·
Erick Ramirez avatar image
Erick Ramirez answered rajivbandi_122971 commented

I'm not aware of any plans to upgrade the Java driver in version 2.5 of the Spark connector. I've done a quick scan of the tickets in the queue and didn't find anything relevant.

Is there a specific feature/fix in the Java driver 4.9 release that you're after? If you tell us what problem you're trying to solve, we might be able to provide you a solution.

In the meantime, I'm going to reach out to the Analytics team here at DataStax to get them to respond to your post. Cheers!

1 comment Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thank you for quick response, Erick.

The reasons that we are looking at cassandra OSS driver 4.9.0 are

1) we have an issue that's related to below issue fix in 4.8.0

  • [bug] JAVA-2846: Give system properties the highest precedence in DefaultDriverConfigLoader

2) Less security vulnerabilities as always

0 Likes 0 ·