Hello
I am trying to use spark Cassandra connector to load some data from a table like in the following:
val conf = new SparkConf(true)
.set("spark.cassandra.connection.host", "localhost")
.set("spark.cassandra.auth.username", "test")
.set("spark.cassandra.auth.password", "test")
val spark = SparkSession.builder
.appName("Datastax Scala example")
.master("local[*]")
.config(conf)
.getOrCreate
val rdd = spark.sparkContext.cassandraTable("test", "orders")
rdd.where("id=?", 123).collect.foreach(println)
By I am getting this exception:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:66)
Caused by: java.lang.NoSuchMethodError: scala.util.parsing.combinator.Parsers.$init$(Lscala/util/parsing/combinator/Parsers;)V
at com.datastax.spark.connector.util.CqlWhereParser$.<init>(CqlWhereParser.scala:6)
at com.datastax.spark.connector.util.CqlWhereParser$.<clinit>(CqlWhereParser.scala)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.$anonfun$containsPartitionKey$2(CassandraTableScanRDD.scala:410)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:241)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:241)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:238)
at scala.collection.immutable.List.flatMap(List.scala:352)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.containsPartitionKey(CassandraTableScanRDD.scala:410)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.partitionGenerator$lzycompute(CassandraTableScanRDD.scala:224)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.partitionGenerator(CassandraTableScanRDD.scala:223)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.getPartitions(CassandraTableScanRDD.scala:272)
at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
Without the where("id=?", 123), it is working fine.
In my maven pom, I have these dependencies:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>2.4.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.12</artifactId>
<version>2.4.2</version>
</dependency>
<dependency>
<groupId>com.datastax.spark</groupId>
<artifactId>spark-cassandra-connector_2.12</artifactId>
<version>2.4.2</version>
</dependency>
am I missing something?
