写信给Kafka追随者节点
是否可以写入Kafka追随者节点？我之所以这么问，是因为我们有时会遇到以下情况 - 一个包含经纪人的特定主机，该经纪人是某些分区的领导节点，从生产者主持人中无法访问，同时实际上保持不变（即生产者和领导者主机之间的网络问题），因此新领导人没有当选。
为了创建一个人为的示例，可以使用防火墙阻止主机或端口。在我人为的示例中，我已经
h0: has seven brokers running on ports 9092 - 9098
h1: 3 brokers running between 9092 - 9094
h2: 3 brokers running between 9092 - 9094
h3: 3 brokers running between 9092 - 9094
阻止了传出的端口9092，正如预期的那样，大约25％的消息未能写出，而错误则是预期的。
在现实世界中，我看到一位主人无法到达生产商主持人约5分钟。
有什么方法可以确保将消息写入Kafka群集？
Is it at all possible to write to a kafka follower node? I ask this because we sometimes encounter the following situation - a particular host containing a broker which is the leader node for some partitions becomes inaccessible from the producer host, while actually remaining up (i.e. a network issue between producer and a leader host), so a new leader is not elected.
To create a contrived example, one can block a host or a port using a firewell. In my contrived example, I have
h0: has seven brokers running on ports 9092 - 9098
h1: 3 brokers running between 9092 - 9094
h2: 3 brokers running between 9092 - 9094
h3: 3 brokers running between 9092 - 9094
I blocked outgoing port 9092, and as expected, approximately 25% of the messages do not get written, and error out, as expected.
In the real world, I have seen a host being unreachable for ~5 minutes from the producer host.
Is there any way to ensure that the message gets written to the kafka cluster?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
从 kafka协议详细信息：
您可以提出富有想象力的解决方案，例如建立一个完全独立的Kafka群集，如果主要群体无法访问，并且将Mirrormaker克隆从次级Kafka到主grone，或将数据转发到具有连接性的其他主机如果您的用例确实需要它，则将其用于Kafka群集。但是，大多数选项似乎有些令人费解和昂贵……
也许最好只是缓冲数据并等待连接或调查并投资于改进网络，因此有更多的冗余，并且很难在主机之间拥有网络分区和kafka群集。
It is not possible to produce messages to a follower, from the Kafka protocol details:
You can come up with imaginative solutions like setting up a completely independent Kafka cluster, produce there if the main one is inaccessible and have MirrorMaker clone the data from the secondary Kafka to the main one, or forward the data to some other host that has connectivity to the Kafka cluster if your use-case really requires it. But well, most options seems a bit convoluted and costly...
Maybe is better to just buffer the data and wait to have connectivity back or investigate and invest in improving the network so there is more redundancy and becomes harder to have network partitions between hosts and the Kafka cluster.