I'm trying to set row level ttl on a table as dataframe api doesn't support row level ttl, I'm using rdd's to set the ttl. I'm trying to connect to a cluster and set the cluster config like host address, username, password etc in the spark conf./ and when I try to connect to the cluster it's not honoring the cluster which is set in spark conf.
This is what I'm using
rdd.saveToCassandra(keyspace, table, writeConf = WriteConf(ttl = TTLOption.perRow(ttlColumn)))
and is there a way I can pass the whole spark conf as this write along with the ttl.
Any help would be appreciated.