Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

ROBO avatar image
ROBO asked smadhavan commented

Which version of the Spark connector is compatible with Spark 3.0.3 and Scala 2.12?

Which version of the Spark connector is compatible with Spark 3.0.3 and Scala 2.12

can you please provide best complete maven dependency .

or is it ok to use one jar as maven dependency spark-cassandra-connector-assembly in production code.

spark-cassandra-connector
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

Erick Ramirez avatar image
Erick Ramirez answered

Spark connector 3.0 and 3.1 are compatible with Apache Spark 3.0. See the compatibility matrix on GitHub for details.

To specify the v3.1 dependencies, use:

libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector_2.12" % "3.1.0"

For more info, see the Quick Start Guide. Cheers!

Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

smadhavan avatar image
smadhavan answered smadhavan commented

@ROBO for Apache Spark 3.0.3 compatibility, I would leverage https://mvnrepository.com/artifact/com.datastax.spark/spark-cassandra-connector-assembly_2.12/3.0.1 version of the Spark Cassandra Connector (SCC). Beginning with the SCC version 3.1 we support Apache Spark 3.1.


https://github.com/datastax/spark-cassandra-connector#version-compatibilitySCC Compatibility Matrix Source



1638972538514.png (62.7 KiB)
2 comments Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

ROBO avatar image ROBO ·
is one jar is enough for spark connector?
0 Likes 0 ·
smadhavan avatar image smadhavan ♦ ROBO ·

Yes @ROBO, that single spark-cassandra-connector-assembly jar is sufficient.

0 Likes 0 ·