kafka Connect Confluent S3水槽连接器：io.confluent.connect.avro.avroconverter无法找到

发布于 2025-02-03 09:28:52 字数 1950 浏览 3 评论 0 原文

使用此kafka连接器：

我将其手动安装到我的kafka connect docker image的/opt/opt/kafka/plugins 。我的目的是使用Kafka Connect将AVRO记录从Kafka主题编写为S3。

在运行时，使用Kafka Connect，我会遇到错误：

ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) [main]
org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.connect.avro.AvroConverter for configuration key.converter: Class io.confluent.connect.avro.AvroConverter could not be found.
    at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:744)
    at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:490)
    at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:483)
    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:129)
    at org.apache.kafka.connect.runtime.WorkerConfig.<init>(WorkerConfig.java:452)
    at org.apache.kafka.connect.runtime.distributed.DistributedConfig.<init>(DistributedConfig.java:405)
    at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:95)
    at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)

在Confluent开源中戳了该班级并查看其他Confluent Kafka Connect插件后，我的解决方案是复制 kafka-connect-avro-converter-7.1。 1.JAR 来自Confluent Kafka Connect Avro Converter插件：

https://www.confluent.io/hub/confluentinc/kafka-connect-avro-converter

进入 confluentinc-kafka-connect-s3*/lib 目录。

这似乎很有用，但似乎是骇人听闻的，似乎应该有一种更好的支持，更简单的方法来使它起作用。有更好的方法吗？

原文

With this Kafka Connect Connector:

https://www.confluent.io/hub/confluentinc/kafka-connect-s3

I manually install that into /opt/kafka/plugins of my Kafka Connect Docker image.
My purpose is to use Kafka Connect to write Avro records from Kafka topics to S3.

At runtime, with Kafka Connect, I get the error:

ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) [main]
org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.connect.avro.AvroConverter for configuration key.converter: Class io.confluent.connect.avro.AvroConverter could not be found.
    at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:744)
    at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:490)
    at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:483)
    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:129)
    at org.apache.kafka.connect.runtime.WorkerConfig.<init>(WorkerConfig.java:452)
    at org.apache.kafka.connect.runtime.distributed.DistributedConfig.<init>(DistributedConfig.java:405)
    at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:95)
    at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)

After poking around for that class in Confluent open source, and looking at other Confluent Kafka Connect plugins, my solution is to copy kafka-connect-avro-converter-7.1.1.jar from the Confluent Kafka Connect Avro Converter plugin here:

https://www.confluent.io/hub/confluentinc/kafka-connect-avro-converter

into the confluentinc-kafka-connect-s3*/lib directory.

This seems to work, but it seems hacky, and it seems like there probably should be a better supported, simpler way of getting this to work. Is there a better way?

分享到QQ

分享到微博