Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

wdeng avatar image
wdeng asked ·

spark Cassandra connector 的df.save()写入Cassandra速度很慢,有什么调优的手段吗?

performancewritespark cassandra connector
1584410837270.png (87.5 KiB)
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

wdeng avatar image
wdeng answered ·

Spark Cassandra Connector的作者Russell Spitzer曾经做过一个讲座https://www.youtube.com/watch?v=cKIHRD6kUOc,slides在这里:https://www.slideshare.net/DataStax/maximum-overdrive-tuning-the-spark-cassandra-connector-russell-spitzer-datastax-c-summit-2016

里面有讲到不少调优的窍门,比如:Batching Key,Sorting,Turning off Batching When Beneficial,Having enough Data in a Task。

另外,这个开源项目的参考手册里提到的这些参数可以看一下:https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md#write-tuning-parameters

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.