How can I access nested collections inside a UDF?

Question

question

andreasrimmelspacher_189038 asked May 8, '20 andreasrimmelspacher_189038 commented May 9, '20

How can I access nested collections inside a UDF?

@alexandre.dutra may I ask another question concerning the topic... I am struggling figuring out weather my setup is wrong, my syntax is wrong or if it's just not possible. How can I access nested collections, e.g.

  
                 
                  values list<frozen <map<int, int>>>

I tried:

  
                 
                  List<Map<Integer, Integer>> myList = state.getList(5 , Map.class);

Leading to:

  
                 
                  Type mismatch: cannot convert from List<Map> to List<Map<Integer,Integer>>

I tried:

  
                 
                  List<Map<Integer, Integer>> myList = state.getList(5, TypeTokens.mapOf(Integer.class, Integer.class));

Leading to:

  
                 
                  TypeTokens cannot be resolved

I am experimenting with Cassandra 3.11.6 and DSE 6.8.0.

I am kind of desperate here...

driver java

7

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

alexandre.dutra ♦ commented · May 08, 2020 at 01:24 PM

The correct version is with TypeTokens. I'm very surprised by the error you got. Are you sure you are importing the com.datastax.driver.core.TypeTokens class? Did you recompile the driver?

0 ·

andreasrimmelspacher_189038 commented · May 08, 2020 at 01:42 PM

That's what I mean, I am starting to learn Cassandra from scratch and I start doubting my setup.

When I check DSE 6.8.0 folder (installation via binary tarball) I find resources/driver/lib/dse-java-driver-core-1.10.0-dse+20200217.jar. In this .jar I find com.datastax.driver.core.TypeTokens.class.

Furthermore I just skript in cql-shell.

So back to your question: How can I import the class in cql and how can I recompile the driver?

0 ·

alexandre.dutra ♦ andreasrimmelspacher_189038 commented · May 08, 2020 at 01:48 PM

Import in cql? Sorry I'm getting lost. What are you trying to do exactly? Is the code above from a Java application, or is it from a cqlsh shell session? I think you need to start reading the driver docs now, because you seem to be trying things that developers are not supposed to do, like inspecting the jars contained in DSE 6.8. If you are using Java, you can start reading from here. This page has a section about retrieving collection types. BTW you should upgrade to the latest Java driver 4.6.

0 ·

alexandre.dutra ♦ alexandre.dutra ♦ commented · May 08, 2020 at 01:51 PM

Sorry I forgot to mention: in old driver versions the class is called TypeTokens, but in 4.6 it is now named GenericType and the method you are looking for is GenericType.mapOf(). Good luck!

0 ·

Show more comments

DataStax Enterprise is powered by the best distribution of Apache Cassandra ™

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Privacy Policy Terms of Use

Answer 1 · 2020-05-08T16:40:37Z

alexandre.dutra answered May 8, '20

You must understand that UDF code is compiled and executed in a sandboxed environment, for security reasons.

Because of that, not all Java classes are available inside the UDF body. The official Cassandra documentation explains it, see here.

So in short, Guava's TypeToken API is not available inside an UDF. This makes your UDF very hard to implement, and is a sign that UDFs were not designed to be so complex.

However if you really need it, there is a workaround:

CREATE OR REPLACE FUNCTION my_function (state my_type, name text)
CALLED ON NULL INPUT
RETURNS my_type
LANGUAGE java
AS
$$
  List<Map<Integer, Integer>> myList = (List) state.getObject(5);
  //do something to myList and put it back in state
  state.setList(5, myList);
  return state;
$$;

By using the driver's geObject() method, you avoid the need to reference the TypeToken API.

Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

question