奇怪的 Kafka 消费者组重新平衡
我的 kafka 主题有两个分区和单个 kafka 消费者。我已经在 AWS 中部署了我的消费者应用程序（spring kafka）。在日志中我看到卡夫卡消费者不时地重新平衡。这种情况并不常见。根据当前的观察，当消费者正在收听主题并空闲时，会发生这种重新平衡。如果有人能向我解释这种行为，我将不胜感激。我在这里发布了一些日志。
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Request joining group due to: group is already rebalancing
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Revoke previously assigned partitions order-response-v3-qa-0, order-response-v3-qa-1
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Revoke previously assigned partitions order-response-v3-qa-0, order-response-v3-qa-1
b2b-group: partitions revoked: [order-response-v3-qa-0, order-response-v3-qa-1
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] (Re-)joining group
My kafka topic has two partitions and single kafka consumer. I have deployed my consumer application(spring kafka) in the AWS. In logs I see kafka consumer re-balance in time to time. This is not frequent. As per the current observation when consumer is listening to the topic and idle this re-balancing occurs. Appreciate if someone can explain me this behavior. I have posted some logs here.
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Request joining group due to: group is already rebalancing
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Revoke previously assigned partitions order-response-v3-qa-0, order-response-v3-qa-1
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] Revoke previously assigned partitions order-response-v3-qa-0, order-response-v3-qa-1
b2b-group: partitions revoked: [order-response-v3-qa-0, order-response-v3-qa-1
[Consumer clientId=consumer-b2b-group-1, groupId=b2b-group] (Re-)joining group
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
重新平衡是一项自动优化不均匀工作负载以及拓扑更改（例如添加或删除代理）的功能。这是通过后台进程实现的，该进程不断检查各种指标以确定是否以及何时应该进行重新平衡。
您可以通过以下链接获取更多知识：
https://medium.com/streamthoughts/apache-kafka-rebalance-protocol-or-the-magic-behind-your-streams-applications-e94baf68e4f2
Re-balancing is a feature that automatically optimizes uneven workloads as well as topology changes (e.g., adding or removing brokers). This is achieved via a background process that continuously checks a variety of metrics to determine if and when a to rebalance should occur.
you can go through the below link for further knowledge:
https://medium.com/streamthoughts/apache-kafka-rebalance-protocol-or-the-magic-behind-your-streams-applications-e94baf68e4f2
如果消费者加入或离开组，Kafka 就会开始重新平衡。以下是可能或将会发生这种情况的各种原因。
消费者加入组：
消费者离开群组：
session.timeout.ms 超出 — 没有发送心跳，可能是因为应用程序崩溃或网络错误
如果您想更深入地了解。这是我读过的一篇令人惊叹的文章
https:// medium.com/bakdata/solving-my-weird-kafka-rebalancing-problems-c05e99535435
Kafka starts a rebalancing if a consumer joins or leaves a group. Below are various reasons why that can or will happen.
A consumer joins a group:
A consumer leaves a group:
session.timeout.ms exceeded — no heartbeats sent, likely because of an application crash or a network error
If you would like to understand more in depth. Here is one of the amazing article I have read
https://medium.com/bakdata/solving-my-weird-kafka-rebalancing-problems-c05e99535435