question

francisdomoney@aol.com avatar image
francisdomoney@aol.com asked Erick Ramirez edited

Where do I install the Kafka connector?

I am installing on Ubuntu 20.04.

Is there any guidance as to where kafka_plugins_dir might be and what it is called?

The instructions say

Configured the DataStax connector JAR using one of the following methods:

  • Move the DataStax connector JAR to the Kafka plugins directory:
    mv installation_location/kafka-connect-cassandra-sink-1.4.0.jar kafka_plugins_dir
  • Configure the path to the JAR:Apache Kafka 0.11.x and later - Specify the JAR location in the plugin.path parameter in the connect-standalone.properties or connect-distributed.properties file that is passed to the worker start-up scripts. Example:
    plugin.path=install_location/kafka-connect-cassandra-sink-1.4.0.jar
kafka-connector
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

Tomasz Lelek avatar image
Tomasz Lelek answered francisdomoney@aol.com commented

Hello, you need to locate the connect-standalone.properties or connect-distributed.properties - you should pick one file, depending on the mode in which you want to run your connector (standalone or distributed).

Next, if you are using a standard confluent platform (https://www.confluent.io/download), those files should be in the /etc/kafka/ directory.

Regarding kafka_plugins_di - the kafka-connect-cassandra-sink-1.4.0.jar can be in any directory. If you are using the confluent platform, you can put the jar in the /share/java/

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

francisdomoney@aol.com avatar image francisdomoney@aol.com commented ·

Thank you. I am in New Major Application Hell, where you can spend hours debugging if you put something in the wrong place

0 Likes 0 ·
Cedrick Lunven avatar image
Cedrick Lunven answered francisdomoney@aol.com commented

Hi @francisdomoney@aol.com

As per kafka documentation here:

https://docs.confluent.io/current/connect/userguide.html#connect-installing-plugins

Kafka Connect finds the plugins using a plugin path defined as a comma-separated list of directory paths in the plugin.path worker configuration property. The following shows an example plugin.path worker configuration property:

plugin.path=/usr/local/share/kafka/plugins

To install a plugin, place the plugin directory or uber JAR (or a symbolic link that resolves to one of these) in a directory already listed in the plugin path. Or, you can update the plugin path by adding the absolute path of the directory containing the plugin. Using the plugin path example above, you would create a /usr/local/share/kafka/plugins directory on each machine running Connect and then place the plugin directories (or uber JARs) there.

Cheers

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

francisdomoney@aol.com avatar image francisdomoney@aol.com commented ·

Thank you. I am in New Major Application hell, where you can spend hours debugging if you put something in the wrong place

0 Likes 0 ·