Please share your comments on the following:
We have an application that needs unique value (2 byte integer) for a transaction.
This application is multithreaded and multiple instances are run parallelly. Each application or thread should have unique transactionid. Application will check the current value in Cassandra, increment it by one and using LWT updates the value in column.
As multiple instances are updating the same column, SERIAL consistency is used to make sure transaction ID is unique.
Because of this LWT and SERIAL consistency we are having the following issues:
1. Latency - A single LWT is getting failed multiple times to get one unique transaction.
2. DC failover - If a node or a DC goes down, because of serial consistency transaction's are getting failed in Cassandra.
6 Node cluster --> 2 Data centers each with 3 nodes and replication factor of 2. Each DC have 3 client applications.
How to achieve low latency and handle failover?