Kafka SASL_SSL 没有名为“Client”的 JAAS 配置部分；在指定的 JAAS 配置文件中找到

Question

我正在尝试在我的 kafka 代理中使用 SASL/PLAIN 激活身份验证。

JAAS 配置文件如下

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret"
    user_alice="alice-secret";
};

我使用以下命令启动 kafka 服务

export KAFKA_OPTS="-Djava.security.auth.login.config=<PATH>kafka_server_jaas.conf
/bin/kafka-server-start.sh /config/server.properties

kafka 服务未正确启动，我在日志中收到这些错误

javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/home/kafka/kafka/config/kafka_server_jaas.conf'.
    at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:189)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1161)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1210)
[2022-03-16 12:13:16,587] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,588] ERROR [ZooKeeperClient Kafka server] Auth failed, initialized=false connectionState=CONNECTING (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,592] INFO Socket connection established, initiating session, client: /127.0.0.1:46706, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,611] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100002dd98c0000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,612] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,752] INFO [feature-zk-node-event-process-thread]: Starting (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2022-03-16 12:13:16,786] INFO Feature ZK node at path: /feature does not exist (kafka.server.FinalizedFeatureChangeListener)
[2022-03-16 12:13:16,788] INFO Cleared cache (kafka.server.FinalizedFeatureCache)
[2022-03-16 12:13:16,957] INFO Cluster ID = 6WTadNCMRAW4dHoc_JUnIg (kafka.server.KafkaServer)
[2022-03-16 12:13:16,968] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID 6WTadNCMRAW4dHoc_JUnIg doesn't match stored clusterId Some(RJXzPwJeRfawIa_yA0B26A) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
    at kafka.server.KafkaServer.startup(KafkaServer.scala:228)
    at kafka.Kafka$.main(Kafka.scala:109)
    at kafka.Kafka.main(Kafka.scala)

我已经将以下行添加到 server.properties

listeners=SASL_SSL://localhost:9092
security.protocol=SASL_SSL
security.inter.broker.protocol=SASL_SSL
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN

Answer 1

[2022-03-16 12:13:16,968] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID 6WTadNCMRAW4dHoc_JUnIg doesn't match stored clusterId Some(RJXzPwJeRfawIa_yA0B26A) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
    at kafka.server.KafkaServer.startup(KafkaServer.scala:228)
    at kafka.Kafka$.main(Kafka.scala:109)
    at kafka.Kafka.main(Kafka.scala)

当代理的 Zookeeper 和 Kafka 数据目录中存储的集群 ID 不匹配时，就会出现此问题。

中存储的代理的集群 ID

• 在本例中， Zookeeper 数据
• 为 6WTadNCMRAW4dHoc_JUnIg Kafka meta.properties 为 RJXzPwJeRfawIa_yA0B26A

原因：
Zookeeper 数据目录被删除。

删除 Zookeeper dataDir 并重新启动 Zookeeper 和 Kafka 服务将不起作用。因为 Zookeeper 会创建一个新的集群 ID，并在注册时并且如果还没有条目的话将其分配给代理。这个新的集群 ID 将与 meta.properties 中的集群 ID 不同。

可以通过以下步骤解决此问题：

• 删除 Kafka log.dirs 和 Zookeeper dataDir - 导致数据丢失； Kafka 和 Zookeeper 服务都需要重启
• 删除 Kafka log.dirs 目录中的 meta.properties - 不会丢失数据；无论如何，都需要启动 Kafka 服务，
• 使用 Zookeeper 数据中存储的值更新 meta.properties 中的集群 ID；在这种情况下，将 RJXzPwJeRfawIa_yA0B26A 替换为 6WTadNCMRAW4dHoc_JUnIg - 不会丢失数据；无论如何，都需要启动 Kafka 服务

javax.security.auth.login.LoginException: No JAAS 配置部分名为 'Client' 在指定的 JAAS 配置文件中找到

Client 部分是用于验证与 ZooKeeper 的 SASL 连接。上面的错误 javax.security.auth.login.LoginException 是一个警告，如果 Zookeeper 允许，Kafka 将在没有 SASL 身份验证的情况下连接到 Zookeeper 服务器。

[2022-03-16 12:13:16,587] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,588] ERROR [ZooKeeperClient Kafka server] Auth failed, initialized=false connectionState=CONNECTING (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 12:13:16,592] INFO Socket connection established, initiating session, client: /127.0.0.1:46706, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 12:13:16,611] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100002dd98c0000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)

KafkaServer 部分由 Broker 使用，并为 Broker 间连接提供 SASL 配置选项。代理使用用户名和密码来启动与其他代理的连接。 user_username 属性集定义连接到代理的所有用户的密码。