As you mentioned a consumer offset can also be managed manually which you will need in this case to avoid duplications(by default the delivery guarantee is at least once) or missing an email as mentioned in your use case.
To manually control the offset auto-commit should be disabled (enable.auto.commit = false).
Now regarding the second part of your question where you mention:
if we don't commit the read-offset then the same message is sent to multiple consumers
This understanding is not fully correct. In Kafka by default each consumer controls its own offset and consumers in the same consumer group don't share partitions(each consumer in the same consumer group reads from a single partition) so the message will not be processed (in Kafka consumers poll messages) by other consumers in the same consumer group, that would happen only to consumers in a different consumer group but then it would be the intention that they also read the message anyway, that's by design in Kafka clients. It's also relevant that you understand that the consumer poll returns multiple messages and the default commitSync or commitAsync will commit the offset of all messages returned by that poll call if you want to avoid possible email duplication you might want to use a more specific commit, check the API here: https://kafka.apache.org/30/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
发布评论
评论（1）
正如您提到的，也可以手动管理消费者偏移量，在这种情况下，您将需要该偏移量（默认情况下，交付保证至少一次）或缺少用例中提到的电子邮件。
为了手动控制偏移量的自动信号（enable.auto.commit = false）。
现在，关于您提到的问题的第二部分：
这种理解并不完全正确。在默认情况下，在kafka中，每个消费者都会控制自己的偏移，而消费者则在同一消费者组中 不共享分区（同一消费者组中的每个消费者都从一个分区中读取），因此不会处理该消息（在Kafka消费者民意测验中）由同一消费者群体中的其他消费者进行，这只会发生在其他消费者群体中的消费者中，但这是他们的意图，即他们还是无论如何都会阅读该消息，这是由Kafka客户端设计的。您还了解，您了解消费者民意调查会返回多条消息，默认情况下consync或commitsync将承担该民意调查返回的所有消息的偏移，如果您想避免使用可能的电子邮件重复，则可能想使用更具体的提交，请检查此处的API：
当您学习这是一种常见的误解，并且有一些不错的资源来理解，并且有一些不错的资源请阅读文档的正式设计部分： htttps：// htttps：// kafka.apache.org/documentation/#theconsumer 另外此免费书籍章节： https://www.oreilly.com/library/view/kafka-the-definistive/9781491936153/ch04.html 特定于您现在所做的事情。祝你好运。
As you mentioned a consumer offset can also be managed manually which you will need in this case to avoid duplications(by default the delivery guarantee is at least once) or missing an email as mentioned in your use case.
To manually control the offset auto-commit should be disabled (enable.auto.commit = false).
Now regarding the second part of your question where you mention:
This understanding is not fully correct. In Kafka by default each consumer controls its own offset and consumers in the same consumer group don't share partitions(each consumer in the same consumer group reads from a single partition) so the message will not be processed (in Kafka consumers poll messages) by other consumers in the same consumer group, that would happen only to consumers in a different consumer group but then it would be the intention that they also read the message anyway, that's by design in Kafka clients. It's also relevant that you understand that the consumer poll returns multiple messages and the default commitSync or commitAsync will commit the offset of all messages returned by that poll call if you want to avoid possible email duplication you might want to use a more specific commit, check the API here: https://kafka.apache.org/30/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
As you're learning this is a common misunderstanding and there are some nice resources to read that clarify this concept for free around, I suggest the official design section of the documentation: https://kafka.apache.org/documentation/#theconsumer also this free book chapter: https://www.oreilly.com/library/view/kafka-the-definitive/9781491936153/ch04.html is specific to what you're doing now. Good luck.