错误 3.10 Kafka 中锁定目录 /var/kafka-logs 时出现磁盘错误

发布于 2025-01-18 01:35:15 字数 1525 浏览 3 评论 0原文

我正在使用Kafka 3.1.0,Portainer 2.9.0和Docker 20.10.11来构建1个经纪人,1个消费者和1个生产商集群。 我试图通过从容器到主机计算机的Docker-Compose绘制日志DIR,以持续该目录的内容(因为如果容器掉落了该信息,则该信息将丢失)。我知道建议拥有超过1个经纪人,但是由于我只是在测试此功能,所以我不想让自己过度复杂化。 我遇到的问题是

 ERROR Disk error while locking directory /var/kafka-logs (kafka.server.LogDirFailureChannel)
 java.nio.file.AccessDeniedException: /var/kafka-logs/.lock
[2022-03-31 12:00:53,986] ERROR [KafkaServer id=1] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)

我已经检查过,执行代理的用户具有所有权限(因为我使用Dockerfile创建了该目录)。

RUN mkdir /var/kafka-logs \
  && chown -R kafka:kafka /var/kafka-logs \
  && chmod -R 777 /var/kafka-logs

我已经看到这个问题在3.0版本中是一件事情,并且已修复在3.1中,而且它仅发生在Windows中,因此我不知道此问题的来源。

编辑:我已经检查了,即使没有映射,它仍然会打印出该错误。它必须是将log.dirs属性更改为非 /tmp目录的问题,因为如果我留下默认配置,则可以正常工作。 默认情况下,我的意思是:

log.dirs=/tmp/kafka-logs

我的码头组合:

version: "3.8"

networks:
    net:
        external: true

services:

  kafka-broker1:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    volumes:
      - /var/volumes/kafka/config/server1.properties:/opt/kafka/config/server.properties
    networks:
      - net

 kafka-producer:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    stdin_open: true
    tty: true
    networks:
      - net
   
  kafka-consumer:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    stdin_open: true
    tty: true
    networks:
      - net

I am using Kafka 3.1.0, Portainer 2.9.0 and docker 20.10.11 to build a 1 broker, 1 consumer and 1 producer cluster.
I am trying to map the log dirs via the docker-compose from the container to the host machine in order to persist the content of that directory (because if the container falls that information will be lost). I know it is recommended to have more than 1 broker, but since I am just testing this feature, I don't want to overcomplicate myself.
The problem I get is

 ERROR Disk error while locking directory /var/kafka-logs (kafka.server.LogDirFailureChannel)
 java.nio.file.AccessDeniedException: /var/kafka-logs/.lock
[2022-03-31 12:00:53,986] ERROR [KafkaServer id=1] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)

I have checked and the user that executes the broker has all permissions (since I created that directory with my Dockerfile).

RUN mkdir /var/kafka-logs \
  && chown -R kafka:kafka /var/kafka-logs \
  && chmod -R 777 /var/kafka-logs

I have seen that this problem was a thing in the 3.0 version and was fixed in the 3.1, and also that it only happened in Windows, so I don't know the source of this problem.

Edit: I have checked and even without the mapping it still prints that error. It must be a problem of changing the log.dirs property to a non /tmp directory, because if I leave the default configuration it works just fine.
By default I mean the following:

log.dirs=/tmp/kafka-logs

My docker-compose:

version: "3.8"

networks:
    net:
        external: true

services:

  kafka-broker1:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    volumes:
      - /var/volumes/kafka/config/server1.properties:/opt/kafka/config/server.properties
    networks:
      - net

 kafka-producer:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    stdin_open: true
    tty: true
    networks:
      - net
   
  kafka-consumer:
    image: registry.gitlab.com/repo/kafka:2.13_3.1.0_v0.1
    stdin_open: true
    tty: true
    networks:
      - net

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

笑脸一如从前 2025-01-25 01:35:15

问题在于我一直在创建一些Docker图像和具有相同名称的容器,并且没有选择最新的图像。

一旦我删除了其余的图像,并且容器选择了最后的所有功能都很好,因此基本上是一个问题,即没有足够的权限来获得该目录的锁定。

The problem was that I have been creating a few docker images and the container with the same name and it didn't picked the newest image.

Once I erased the rest of images and the container picked the lastest it all worked just fine, so it was basically a problem of not having enough permissions to get the lock of that directory.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文