We have some keyspaces with 2 replicas that we query with LOCAL_ONE to tolerate node failures.
Now we're about to upgrade those to have 3 replicas and LOCAL_QUORUM but I struggle to find a process that works.
These are my findings, and I guess questions. Please correct me if I got something wrong.
- Doing the alter first means 1 out of 3 nodes does not have the data until "nodetool repair --full" finishes. Quering that node will result in the data not being found.
- Changing to LOCAL_QUORUM first puts us at risk of losing availability until the alter has happend.
Is there some other way of doing this?
Is it weird that the new node accepts reads before it has been correctly repaired?