question

igor.rmarinho_185445 avatar image
igor.rmarinho_185445 asked Erick Ramirez edited

Dropping empty MVs taking a long time

Hi,

Some of the DEV team is complaining about dropping empty MV is taking a long time,

The error they are reporting is:

Caused by: com.datastax.driver.core.exceptions.OperationTimedOutException: [/10.30.50.3:9042] Timed out waiting for server response
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:954)
at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1575)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:670)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:745)

When I check the logs I just see the messages below:

INFO  [DSE main thread] 2020-06-08 15:15:48,966  Server.java:219 - Effective settings: Netty Epoll = true, AIO = true, data directories on SSD = true, system RAM = 25.355GiB, Max Direct Mem = 10.000GiB, file cache size = 5.000GiB
INFO  [DSE main thread] 2020-06-08 15:15:48,966  Server.java:226 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
INFO  [DSE main thread] 2020-06-08 15:15:48,973  PluginManager.java:326 - Activating plugins which require native transport
INFO  [DSE main thread] 2020-06-08 15:15:48,974  PluginManager.java:333 - Activation of plugins which require native transport finished
INFO  [DSE main thread] 2020-06-08 15:15:49,147  DseDaemon.java:815 - DSE startup complete.
INFO  [CoreThread-2] 2020-06-08 15:15:55,043  PercentileFilter.java:117 - CqlSlowLog plugin now warmed up!
INFO  [ValidationSchedulerEventExecutor:5] 2020-06-08 15:16:11,619  NodeSyncService.java:170 - Enabled NodeSync service (470 tables have NodeSync enabled)
INFO  [GCInspector:1] 2020-06-08 15:16:27,053  GCInspector.java:315 - G1 Young Generation GC in 455ms.  G1 Old Gen: 591407616 -> 595601920; G1 Survivor Space: 171966464 -> 360710144; G1 Eden Space: 4114612224 -> 0
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:35,797  DocumentsWriter.java:785 - Waited 140 milliseconds for 1 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:36,347  DocumentsWriter.java:785 - Waited 29 milliseconds for 1 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:40,926  DocumentsWriter.java:785 - Waited 14 milliseconds for 1 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:41,291  DocumentsWriter.java:785 - Waited 0 milliseconds for 0 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:41,704  DocumentsWriter.java:785 - Waited 10 milliseconds for 1 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:46,092  DocumentsWriter.java:785 - Waited 9 milliseconds for 1 flushing segments.
INFO  [commitScheduler-23-thread-1] 2020-06-08 15:16:47,059  DocumentsWriter.java:785 - Waited 11 milliseconds for 1 flushing segments.
INFO  [GCInspector:1] 2020-06-08 15:16:51,082  GCInspector.java:315 - G1 Young Generation GC in 448ms.  G1 Old Gen: 661700344 -> 938425856; G1 Survivor Space: 360710144 -> 125829120; G1 Eden Space: 2315255808 -> 0

I didnt find much information about it, any idea on it? Could it be related to NodeSync ?

I'm using DSE 6.8

schema
8 comments
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

bettina.swynnerton avatar image bettina.swynnerton ♦♦ commented ·

Hi, did you check if the MV was successfully dropped? How many nodes are in your cluster?

0 Likes 0 ·
igor.rmarinho_185445 avatar image igor.rmarinho_185445 bettina.swynnerton ♦♦ commented ·

It drops the table, I have a 3 node cluster.

0 Likes 0 ·
bettina.swynnerton avatar image bettina.swynnerton ♦♦ igor.rmarinho_185445 commented ·

In one of your nodes' system.log you would see that the DDL statement gets executed.

I did a quick test and dropped an MV through cqlsh:

INFO  [CoreThread-0] 2020-06-08 15:35:30,271  QueryProcessor.java:420 - Executing DDL statement 'drop materialized view friends.person__knows__person_by_in_person_id;' from client IP /127.0.0.1:34502 by user #

Perhaps you can see more details around that statement that could help?

0 Likes 0 ·
Show more comments

0 Answers