Which version of the Spark connector is compatible with Spark 3.0.3 and Scala 2.12
can you please provide best complete maven dependency .
or is it ok to use one jar as maven dependency spark-cassandra-connector-assembly in production code.
Bringing together the Apache Cassandra experts from the community and DataStax.
Want to learn? Have a question? Want to share your expertise? You are in the right place!
Not sure where to begin? Getting Started
Which version of the Spark connector is compatible with Spark 3.0.3 and Scala 2.12
can you please provide best complete maven dependency .
or is it ok to use one jar as maven dependency spark-cassandra-connector-assembly in production code.
Spark connector 3.0 and 3.1 are compatible with Apache Spark 3.0. See the compatibility matrix on GitHub for details.
To specify the v3.1 dependencies, use:
libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector_2.12" % "3.1.0"
For more info, see the Quick Start Guide. Cheers!
@ROBO for Apache Spark 3.0.3 compatibility, I would leverage https://mvnrepository.com/artifact/com.datastax.spark/spark-cassandra-connector-assembly_2.12/3.0.1 version of the Spark Cassandra Connector (SCC). Beginning with the SCC version 3.1
we support Apache Spark 3.1
.
SCC Compatibility Matrix Source
Yes @ROBO, that single spark-cassandra-connector-assembly
jar is sufficient.
8 People are following this question.
Is it possible to achieve data locality on level rack_local when using Spark Cassandra Connector?
Is there a way to obtain CassandraConnector from cluster config stored in SparkSession?
Is filtering on 1 column of a composite partition key not allowed?
DirectJoin does not work with Structured Streaming DataFrame
DataStax Enterprise is powered by the best distribution of Apache Cassandra ™
© 2022 DataStax, Titan, and TitanDB are registered trademarks of DataStax, Inc. and its subsidiaries in the United States and/or other countries.
Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.
Privacy Policy Terms of Use