I am reading cassandra table using pyspark job , but it is throwing an error :
Caused by: com.datastax.driver.core.exceptions.TransportException: <ip> Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1210) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1195) at com.datastax.driver.core.Connection.defunct(Connection.java:445)
nodetool csftstas for table:
Space used (live): 21.32 GiB Space used (total): 21.32 GiB Space used by snapshots (total): 0 bytes Off heap memory used (total): 14.96 MiB
I have three nodes cassandra cluster:
CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 2 Socket(s): 1 NUMA node(s): 1
total used free shared buff/cache available Mem: 15 2 0 0 12 11 Swap: 0 0 0
grep MemTotal /proc/meminfo MemTotal: 15923288 kB
When I read data from pyspark it is throwing error , My question is :
are these configs not enough for cassandra node with table size given by nodetool command , so to consume data cassandra does not overload or does not throw an error .
There is no other job running in the cassandra cluster.