question

pihu_0206 avatar image
pihu_0206 asked pihu_0206 published

UnavailableException: Not enough replicas available for query at consistency TWO (2 required but only 1 alive)

I am facing some unexpected issues.

Here you go the detailed information

RF=2
Cluster size=3
Write Consistency= Quorum

Few configurations cassandra.yaml

endpoint_snitch: GossipingPropertyFileSnitch

Keyspace Infirmation

describe keyspace test;

CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '2'}  AND durable_writes = true;

Test Case
1) I keep the one node down from the 3 node cluster. 2 Node is up but

Getting below Error

org.springframework.data.cassandra.CassandraInsufficientReplicasAvailableException: SessionCallback; CQL [INSERT INTO test (some data USING TTL 2592000]; Not enough replicas available for query at consistency TWO (2 required but only 1 alive); nested exception is com.datastax.oss.driver.api.core.servererrors.UnavailableException: Not enough replicas available for query at consistency TWO (2 required but only 1 alive)
at org.springframework.data.cassandra.core.cql.CassandraExceptionTranslator.translate(CassandraExceptionTranslator.java:108) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.cql.CassandraAccessor.translate(CassandraAccessor.java:379) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.cql.CqlTemplate.translateException(CqlTemplate.java:718) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.cql.CqlTemplate.execute(CqlTemplate.java:131) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate.executeSave(CassandraTemplate.java:863) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate.executeSave(CassandraTemplate.java:854) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate.doInsert(CassandraTemplate.java:665) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate.doInsert(CassandraTemplate.java:648) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate.insert(CassandraTemplate.java:629) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraOperations$insert.call(Unknown Source) ~[na:na]
at com.adsizzler.adosiz.kafkaconsumer.clicks.storage.retry.repository.impl.ClickCassandraRepoImpl.saveAsync(ClickCassandraRepoImpl.groovy:106) ~[classes!/:na]
at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_302]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_302]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) [spring-aop-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) [spring-tx-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) [spring-aop-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_302]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_302]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_302]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_302]
Caused by: com.datastax.oss.driver.api.core.servererrors.UnavailableException: Not enough replicas available for query at consistency TWO (2 required but only 1 alive)
at com.datastax.oss.driver.api.core.servererrors.UnavailableException.copy(UnavailableException.java:103) ~[java-driver-core-4.6.1.jar!/:na]
at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) ~[java-driver-core-4.6.1.jar!/:na]
at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53) ~[java-driver-core-4.6.1.jar!/:na]
at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30) ~[java-driver-core-4.6.1.jar!/:na]
at com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230) ~[java-driver-core-4.6.1.jar!/:na]
at com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:53) ~[java-driver-core-4.6.1.jar!/:na]
at org.springframework.data.cassandra.core.CassandraTemplate$StatementCallback.doInSession(CassandraTemplate.java:977) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.CassandraTemplate$StatementCallback.doInSession(CassandraTemplate.java:967) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
at org.springframework.data.cassandra.core.cql.CqlTemplate.execute(CqlTemplate.java:129) ~[spring-data-cassandra-3.0.4.RELEASE.jar!/:3.0.4.RELEASE]
... 20 common frames omitted

As per the log, one node is live so data should be inserted at least in one node and acknowledgement should be received from one node of writing data to DSE. I am surprised why data did not insert into any one live node of the cluster. As I explained above one node is down among the 3 node cluster.

consistency level
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered pihu_0206 commented

You understanding isn't quite correct. Quorum of 2 nodes is still 2 nodes which is why you're getting "2 required but only 1 alive".

But more importantly, you're not writing at QUORUM consistency. You're writing with TWO:

Not enough replicas available for query at consistency TWO

Our general recommendation is to have a minimum of 3 replicas in each DC so your application can tolerate a node outage for consistency levels higher than one. Cheers!

1 comment Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

pihu_0206 avatar image pihu_0206 commented ·

Hi Erick, I understand the problem after reading the logs carefully. I find the solution.

Once again thanks

0 Likes 0 ·