Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

dbyrne_86288 avatar image
dbyrne_86288 asked ·

CDC- Is there sample code available that shows how to read the CDC logs?

We are having issues trying to read the CDC logs generated by DSE 6.7. We are unable to get Schema.instance to initialize properly in order to read the keyspace metadata. We have tried all variations of DatabaseDescriptor.toolInitialization(), clientInitialztion() and daemonInitialization, all with no luck.

cdc
1 comment
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

I'm aware that you were asking about this at the Academy Slack. I just wanted to let you know that I'm looking into this for you. Cheers!

0 Likes 0 · ·

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

@dbyrne_86288 Yuki Morishita (DataStax Principal Architect) did a CDC presentation at this year's DataStax Accelerate. You can watch the recording here -- https://www.datastax.com/accelerate/sessions.

You can access the DSE-specific CDC example project he made on GitHub -- https://github.com/yukim/cassandra-cdc-example/tree/dse. Cheers!

13 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks Eric, I was in the front row for that presentation and have worked with this exact code. It does not work on our DSE install. The Schema.instance is never initialized and is unable to read the keyspaces from our DSE node, it only sees the default schemas. I dont see anything special in Yuki's code to allow for this. Any help would be appreciated. Also this code does not work out of the box as the version for DSE is 600 on the commit logs. The commitlogreader that is packaged in the cassandra-all maven build for 3.11.4 does not allow for this. This code only works for C* , not DSE.

0 Likes 0 · ·

Awesome about being at Accelerate.

I'm sorry that you are having a frustrating time with CDC + DSE. The challenge is that the format is not the same as OSS C* because it was refactored for DSE Advanced Replication. It is part of the reason that there are no public APIs for it. I know that you have a consulting engagement lined up. Perhaps we could prep them for you. Cheers!

0 Likes 0 · ·

Have you got a tarball or package installation? Can you confirm that you are using cassandra/lib/dse-db-all-6.7.3.jar ?

0 Likes 0 · ·

HI, author of the example code here.

Did you try with "dse" branch Erick mentioned? It does not package cassandra-all.jar and just use dse-db.jar in your DSE install.

Also, example code should run with the same user as DSE (typically it is `cassandra` user), since Schema has to be read from the data directory.

3 Likes 3 · ·

OK, I see an issue. When we clone the branch above, we are getting a version that is different then the branch we need. The branch that clones contains the cassandra-all dependencies. We were working from the master branch not the DSE branch

1 Like 1 · ·
Show more comments
Show more comments