Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

wdeng avatar image
wdeng asked Erick Ramirez edited

spark Cassandra connector 的df.save()写入Cassandra速度很慢,有什么调优的手段吗?

performancewrite
1584410837270.png (87.5 KiB)
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

wdeng avatar image
wdeng answered

Spark Cassandra Connector的作者Russell Spitzer曾经做过一个讲座https://www.youtube.com/watch?v=cKIHRD6kUOc,slides在这里:https://www.slideshare.net/DataStax/maximum-overdrive-tuning-the-spark-cassandra-connector-russell-spitzer-datastax-c-summit-2016

里面有讲到不少调优的窍门,比如:Batching Key,Sorting,Turning off Batching When Beneficial,Having enough Data in a Task。

另外,这个开源项目的参考手册里提到的这些参数可以看一下:https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md#write-tuning-parameters

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.