DataStax Academy FAQ

DataStax Academy migrated to a new learning management system (LMS) in July 2020. We are also moving to a new Cassandra Certification process so there are changes to exam bookings, voucher system and issuing of certificates.

Check out the Academy FAQ pages for answers to your questions:


question

hashmiata_77760 avatar image
hashmiata_77760 asked ·

Why shouldn't we worry about PERIODIC-COMMIT-LOG-SYNCER warnings in the logs?

[FOLLOW UP QUESTION TO #1873]

Why shouldn't we worry the warning if it is on its own. I am seeing such warnings frequently in my logs.

WARN [PERIODIC-COMMIT-LOG-SYNCER] 2020-07-07 12:02:56,536 AbstractCommitLogService.java:105 - Out of 179 commit log syncs over the past 302s with average duration of 1282.88ms, 1 have exceeded the configured commit interval by an average of 1.00ms
commitlog
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

Background

The PERIODIC-COMMIT-LOG-SYNCER is a service that fulfils the contract of syncing the commit logs (with fsync) to disk based on these settings in cassandra.yaml:

commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000

Symptom

Warning messages (such as the one you posted) in the Cassandra logs indicate that the interval between fsync calls took longer that the sync period.

Cause

The warnings in earlier versions of Cassandra are false alarms. There was a bug in the way the sync interval was getting calculated in AbstractCommitLogService.java leading to warnings logged incorrectly (CASSANDRA-14451).

Solution

The bug was fixed in Cassandra 3.0.17 and 3.11.3. Upgrade to the latest version to take advantage of improvements and fixes.

In your case, the log entry by the AbstractCommitLogService is from line 105:

WARN [PERIODIC-COMMIT-LOG-SYNCER] ... AbstractCommitLogService.java:105 ...

This indicates that your cluster is running a very old version of Cassandra, most likely version 2.1.21 (or older). In which case, it isn't soo relevant unless you're seeing dropped mutations as well which means the commitlog disk cannot keep up with the writes. When the commitlog disk is overloaded with IO traffic, it can cause the syncs to fall behind. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.