We have created a role in DSE cassandra with login=true and superuser=false. After creation we deleted and recreated the role. We have a six node DSE cluster. We observed that the role has been correctly updated in all nodes except in one node where the can_login and superuser was showing null in system_auth keyspace for the roles table. We manually updated the table to take care of the immediate issue. But we wanted to know why it did not get updated in that node.
Getting "Invalid metadata has been detected for role"
I have found the reason for this. We have a six node cluster. When I recreated the role, it got propagated to all nodes except one node. In that node, the value for can_login and superuser column of the table system_auth.roles was showing as null. So, when the request was going to that node, we were getting this error. After updating the table in that node, everything was fine. I could not however find out why the change was not propagated to that node.
role | can_login | is_superuser | member_of | salted_hash
-----------+-----------+--------------+-----------+--------------------------------------------------------------
xxxxxx | null | null | null | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
This indicates that one of the replicas missed the write and is inconsistent. You shouldn't manually insert data into system tables because they are not designed to be modified directly.
When you run into any inconsistency problems, the correct way of dealing with it is by repairing the affected table or keyspace. Cheers!
7 People are following this question.
