Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

miguel.oyarzo_185223 avatar image
miguel.oyarzo_185223 asked ·

Getting CLUSTERCONFERROR after running an LCM configure job in OpsCenter

From OpsCenter, after running Datacenters/Configure all nodes get the latest configuration and they became healthy. However, in "Lifecycle Manager: Jobs", I got


ERROR - CLUSTERCONFERROR
Cluster connection settings could not be updated: Cannot find cluster configuration in opscenterd. Please update your cluster connection settings manually

(regardless all nodes are Success)

Due to that the cluster shows "Last Job Status Failure"

What's wrong?

------

Stack trace:

slingshot.support$stack_trace.invoke(support.clj:201)
lcm.utils.opscenterd_gateway$call_update_opscd_cluster_config.invokeStatic(opscenterd_gateway.clj:64)
lcm.utils.opscenterd_gateway$call_update_opscd_cluster_config.invoke(opscenterd_gateway.clj:24)
lcm.jobs.multinode.configure.ConfigureJob.on_complete(configure.clj:31)
lcm.jobs.multinode.common$process_complete__11895$fn__11896$fn__11898.invoke(common.clj:262)
lcm.jobs.multinode.common$process_complete__11895$fn__11896.invoke(common.clj:261)
clojure.core.async$thread_call$fn__5990.invoke(async.clj:442)
clojure.lang.AFn.run(AFn.java:22)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)


opscenterlcmjobs
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

craig.mcdaniel avatar image
craig.mcdaniel answered ·

OPSC-16520 will fix this issue moving forward. Look for it in the next 6.5/6.7 release (check release notes).

1 comment Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks, Craig. I'll update this post when the fix gets released. Cheers!

0 Likes 0 · ·
Erick Ramirez avatar image
Erick Ramirez answered ·

@miguel.oyarzo_185223 The error got reported because for whatever reason, LCM failed to update the cluster configuration settings after it applied the changes most likely because the cluster ID doesn't exist anymore. Is it possible that the cluster is still managed by LCM but it is no longer managed by OpsCenter? Another way of putting it is -- did you previously delete the cluster from OpsCenter?

With the limited info available to us, it's a bit difficult to diagnose the problem but see if this KB article applies to your scenario and try the documented workaround. Cheers!

6 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

curl -X PUT http://<my_internal_domain>:8888/api/v2/lcm/clusters/19303340-32ea-46df-8904-590f3560e33e -H "Content-Type: application/json" -d '{"opsc-cluster-id": null}' | json_pp
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/bin/json_pp5.18 line 45

Also,

$ json_pp -v
1.00

any idea?

0 Likes 0 · ·
Erick Ramirez avatar image Erick Ramirez ♦♦ miguel.oyarzo_185223 ·

@miguel.oyarzo_185223 Have you checked if the output of the curl command generates a valid JSON? That's what it's complaining about. Cheers!

0 Likes 0 · ·

@Erick Ramirez

Yes I know :) , it is a PUT command, I understand that

-H "Content-Type: application/json" -d '{"opsc-cluster-id": null}'

is the json injected to OpsCenter API. Yes, '{"opsc-cluster-id": null}' is valid JSON.

Is that what you mean?

this show similar result:


$ curl  -X PUT -H "Content-Type: application/json" -d '{"opsc-cluster-id": null}' http://<my_internal_domain>:8888/api/v2/lcm/clusters/19303340-32ea-46df-8904-590f3560e33e

curl: (3) URL using bad/illegal format or missing URL
curl: (52) Empty reply from server


Cheers

0 Likes 0 · ·
Show more comments

@miguel.oyarzo_185223 I forgot to ask this again:

Is it possible that the cluster is still managed by LCM but it is no longer managed by OpsCenter? Another way of putting it is -- did you previously delete the cluster from OpsCenter?

Because it's possible that the workaround may not apply to your scenario. Cheers!

0 Likes 0 · ·