I'm evaluating using Cassandra as a binary object store. In my experiments I'm not getting good enough read throughput for single queries to single nodes. Even on a MacBook Pro 15 2018 reading e.g. a 128 MiB blob in 64KiB chunks with the Cassandra C++ driver from a single node on the loopback interface takes about 300ms. I was hoping I could saturate a 10 GiB port on a fast enough server with single requests for large blobs. Are there any C++ driver, Cassandra or Java VM tuning options that could improve read throughput in this scenario?
The blob table has the following schema:
CREATE TABLE blobs.blob (
id uuid,
chunk_id int,
blob_size int,
creation_time timestamp,
data blob,
data_type text,
hash blob,
info text,
update_time timestamp,
PRIMARY KEY (id, chunk_id)
) WITH CLUSTERING ORDER BY (chunk_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'enabled': 'false'}
AND crc_check_chance = 0.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
