正确使用kafka全局stateStore查找
很高兴听到一些意见，这是否是一个好主意，因为我以前没有实施这样的事情的经验：
要求：
我有两个主题：主题书籍INFO，主题类别 -信息。 当我从主题书籍中处理消息时，我需要查找来自主题类别INFO的一些元数据。 理想情况下，每个主题都应在单独的应用程序和代码库中处理，因为我们的设计指南（如果它是全球stateStore的限制，则指定此指南）。另外，我们正在使用Kstreams和Spring Kafka。
我有以下想法：
app1：从主题类别INFO中读取消息，并基于特定条件将其保存到全球stateStore中，或将其从Global StateStore中删除。
App2：阅读主题书籍中的消息，在全球stateStore中进行查找以产生最终记录。
这个工作会有什么更好的做法吗？ 感谢您的任何输入！
It would be nice to hear some opinions whether this is a good idea because I don't have previous experience in implementing something like this:
Requirement:
I have two topics: topic book-info, topic category-info.
As I process messages from topic book-info, I need to lookup some metadata which came from topic category-info.
Ideally, each topic should be processed in a separate app and codebase because of our design guidelines (I'm specifying this in case it's a constraint for global statestore). Also, we are using KStreams and Spring Kafka.
I had the following idea:
App1: Read messages from topic category-info and based on specific condition either save them into global statestore or delete them from global statestore .
App2: Read messages from topic book-info, do a lookup in global statestore to produce the final record.
Will this work and are there any better practices?
Thanks for any input!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
老实说，当全球stateStore在一段时间内生长时，尤其是对于长期运行的生产数据管道，除非您在此类Statestore上实现了更好的生命周期，否则它将成为性能问题的瓶颈。
由于您正在为两个不同的源主题设计两个独立的应用程序，因此更好的选择是通过外部DB进行查找以进行记录，并且它将具有更好的控制。或者，您可以使用运行交互式查询的KSQL流/表。
To be honest, global statestore will be a bottle-neck for the performance issue when it grows over a period of time, especially for the long running production data pipeline, unless you have a better lifecycle implemented on such statestore.
Since you are designing two separate applications for two different source topics, better option would be doing a lookup through an external DB for the records and it will be having a better control. Or, you can use the KSQL Streams/Table running interactive queries.