I've spark 3.0.2 cluster in stand alone mode.
I need to aggregate all the nodes logs that we can see in stdout or stderr in one location.
I could see some way to do this in yarn but not in stand alone.
Is there any way instead of using an ELK Stack ?
Bringing together the Apache Cassandra experts from the community and DataStax.
Want to learn? Have a question? Want to share your expertise? You are in the right place!
Not sure where to begin? Getting Started
Is there a way to aggregate all Spark stdout, stderr, logs into one?
When you combine all logs/outputs into one means it would be difficult to determine where each individual message/line came from. What you're asking doesn't make sense to me so perhaps I misunderstood your intent.
In any case, this is very specific to Spark so I'd recommend asking in a Spark forum. Cheers!
8 People are following this question.
