teriksson avatar image
teriksson asked Erick Ramirez commented

What would happen if multiple app sessions make schema changes at the same time?

Reading up on Schema changes / meta data changes / Schema migration

I come accross amongst others this article:


But what if :

  • All nodes does not reply , what happens then ?
    Do we end up rolling back, OR do we end up out of synch ?

I have understood (experienced actually) that we end up out of synch, and that we had to restart all the nodes in the cluster as we did not know which node had the wrong information.

But the larger the cluster becomes where 3 nodes is small, 30 nodes is not big, 300 nodes hmm kind of big.... the more problematic it is to restart all those nodes.

Is there any way to know which node(s) that are not in line, not synched ?

further my understanding is also that the meta data agreement is with ALL, as in ALL DCs, DC1, DC2, DCn, so a rather big synchronisation that needs to happen within 10 seconds.

What if

  • MORE THAN ONE application tries to modify the same Keyspace at the same time or close to the same time ?

Reading the article this should be ok it seems like, but my experience is not that at all.

So does that mean that all applications/clients that need to change the schema / meta data needs to first coordinate this with e.g. global mutex, zookeeper or something similar to ensure that only one tries to do this at the same time ?

java driverschema
1606726863772.png (17.2 KiB)
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

If not all nodes respond or some timeout then your cluster is mostly likely to end up in a schema disagreement -- there is no rollback mechanism that takes place in this scenario. And you are correct that a rolling restart is required to fix it.

For this reason, programatically changing the schema isn't recommended. The feature is provided by the drivers as a convenience but as you already know, there are a lot of problems that can arise from it. Cheers!

2 comments Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

pahlgren avatar image pahlgren commented ·

What is the difference when doing a change "manually" using cqlsh and doing it from an application if the application make sure it only do one change at a time?

Is it okay to do changes "manually" to several keyspaces at the same time?

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ pahlgren commented ·

Making changes through cqlsh means that you are forced to only one DDL statement at a time mostly because you don't get the command prompt back until the previous DDL statement has completed.

And no, you can not perform multiple DDL changes in parallel regardless of which keyspace definition is being altered. Cheers!

0 Likes 0 ·