带有多个水槽的背压控制
在我们的应用程序中,我们想使用Apache Flink处理用户事件, 用不同的水槽类型(每个用户都有几种接收器类型)丰富它们,然后将这些事件沉入相应的不同类型的水槽。
假设我们可能有2种类型的水槽(例如文件和HTTP),每个用户都希望使用其/她自己的HTTP端点/文件名来使用这些类型的一种或两种类型。
丰富之后的每个事件都知道是否应将其发送到单个接收器类型或确切的端点/文件名。
问题在于,某些端点可能会变慢,结果是整个流,所有用户和水槽类型的所有事件都将通过背压减慢。
所以,问题。是否有一种方法可以独立保留合理的背压,对不同的水槽和端点进行处理事件,以使一个用户的缓慢端点不会影响整个流和其他用户。
In our application we would like to use Apache Flink to process user events,
enrich them with different sink types (each user can have several sink types) and then sink these events to corresponding different types of sinks.
Say we may have 2 types of sinks(for example file and http) and each user wants to use one or both of these types, with his/her own http endpoint/file name for corresponding sink.
Each event after enrichment knows if it should be sent to single or both sink types and exact endpoint/ filename.
The problem is that some endpoints may become slower and as the result the entire stream, with all events for all users and sink types will be slowed down by backpressure.
So, the question. Is there a way to, while still keeping reasonable backpressure, process events for different sinks and endpoints independently, so that slow endpoint of one user don't affect the entire stream and other users.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论