We added a DateRangeType field to a UDT, and now our Spark job that reads those records is failing with this error, running in AWS:
Exception in thread “main” java.util.NoSuchElementException: key not found: ‘org.apache.cassandra.db.marshal.DateRangeType’ at scala.collection.MapLike$class.default(MapLike.scala:228
We are running on DSE 5.1.15. Here are the relevant dependencies:
<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.11</artifactId> <version>2.1.1</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.11</artifactId> <classifier>sources</classifier> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-sql_2.11</artifactId> <version>2.1.1</version> <scope>provided</scope> </dependency> <dependency> <groupId>com.datastax.client</groupId> <artifactId>dse-byos_2.11</artifactId> <version>5.1.10</version> <scope>provided</scope> </dependency>
Running locally, we get a different issue:
WARN [dse-app-client-thread-pool-0] 2019-09-17 13:54:59,114 CodecRegistry.java:357 - Ignoring codec DateRangeCodec ['org.apache.cassandra.db.marshal.DateRangeType' <-> com.datastax.driver.dse.search.DateRange] because it collides with previously registered codec DateRangeCodec ['org.apache.cassandra.db.marshal.DateRangeType' <-> com.datastax.driver.dse.search.DateRange]
We can't find the DateRangeType class anywhere on our classpath. What do we need to do?