Build Cloud-Native apps with Apache Cassandra

GOT QUESTIONS from the Workshop? You're in the right place! Post a question here and we'll get you answers.

Click here for Week 8 Materials and Homework.

Follow us on Eventbrite to get notified when new workshops are scheduled!

question

francisdomoney@aol.com avatar image
francisdomoney@aol.com asked ·

Where do I install the Kafka connector?

I am installing on Ubuntu 20.04.

Is there any guidance as to where kafka_plugins_dir might be and what it is called?

The instructions say

Configured the DataStax connector JAR using one of the following methods:

  • Move the DataStax connector JAR to the Kafka plugins directory:
    mv installation_location/kafka-connect-cassandra-sink-1.4.0.jar kafka_plugins_dir
  • Configure the path to the JAR:Apache Kafka 0.11.x and later - Specify the JAR location in the plugin.path parameter in the connect-standalone.properties or connect-distributed.properties file that is passed to the worker start-up scripts. Example:
    plugin.path=install_location/kafka-connect-cassandra-sink-1.4.0.jar
kafka-connector
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

Tomasz Lelek avatar image
Tomasz Lelek answered ·

Hello, you need to locate the connect-standalone.properties or connect-distributed.properties - you should pick one file, depending on the mode in which you want to run your connector (standalone or distributed).

Next, if you are using a standard confluent platform (https://www.confluent.io/download), those files should be in the /etc/kafka/ directory.

Regarding kafka_plugins_di - the kafka-connect-cassandra-sink-1.4.0.jar can be in any directory. If you are using the confluent platform, you can put the jar in the /share/java/

1 comment Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

francisdomoney@aol.com avatar image francisdomoney@aol.com ·

Thank you. I am in New Major Application Hell, where you can spend hours debugging if you put something in the wrong place

0 Likes 0 ·
Cedrick Lunven avatar image
Cedrick Lunven answered ·

Hi @francisdomoney@aol.com

As per kafka documentation here:

https://docs.confluent.io/current/connect/userguide.html#connect-installing-plugins

Kafka Connect finds the plugins using a plugin path defined as a comma-separated list of directory paths in the plugin.path worker configuration property. The following shows an example plugin.path worker configuration property:

plugin.path=/usr/local/share/kafka/plugins

To install a plugin, place the plugin directory or uber JAR (or a symbolic link that resolves to one of these) in a directory already listed in the plugin path. Or, you can update the plugin path by adding the absolute path of the directory containing the plugin. Using the plugin path example above, you would create a /usr/local/share/kafka/plugins directory on each machine running Connect and then place the plugin directories (or uber JARs) there.

Cheers

1 comment Share
10 |1000 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

francisdomoney@aol.com avatar image francisdomoney@aol.com ·

Thank you. I am in New Major Application hell, where you can spend hours debugging if you put something in the wrong place

0 Likes 0 ·