多个并发进行中请求重复数据删除
你能帮助我吗?假设我们有:
- 一个服务
S
,当您给出数字x
的数组时,它将返回数组x
内每个 id 的项目详细信息代码>.假设这是一次昂贵的通话。
现在,您同时收到很多请求。如果每个请求都带有 id,我们可以使用 singleflight 来处理它像下面的代码:
var group singleflight.Group
func GetResult(id int64) (itemDetail, error) {
key := fmt.Sprintf("Key%d",id) // key for identifier the same request id
result, err, _ := group.Do(key, func() (inteface{}, error) {
result, err := GetFromService(id) // Get data from the service S
if err!= nil {
return nil, err
}
return result, nil
}
return result, err
}
现在,如果对于每个请求,都有一个 ids 数组会怎样?有没有办法处理所有请求,使得 x
中的每个 id 只会查询服务 S
一次,而其他重复的 id 将等待某些内容(如果可能)得到结果?
例如,有 3 个并发的正在进行的请求。第一个请求 ID 为 1,2,3
,第二个请求 ID 为 1,2,4
,第三个请求 ID 为 1,3,4
。目标是仅调用服务 S
2 次。第一个调用将请求 id 1,2,3
。本次调用结束后,可以将结果共享给第二个和第三个请求,从而第一个请求完成。
当第一个调用仍在调用服务 S 时,第二个请求到来,从那里我们知道第二个请求只需要请求 id 4,因此随后执行此操作(因为完成后的第一个调用将“共享”第二次调用的结果)。
虽然第一个呼叫和第二个呼叫正在进行中,但我们假设第三个呼叫打来,并且我们确定在第三个呼叫中..除了等待第一个请求和第二个请求之外,不需要执行任何操作完成。在第一个请求和第二个请求完成后,他们将随后分享来自 X
的调用结果并结束第三个请求,
我似乎找不到与此相关的任何问题,所以在检查了这么多之后线程...如果这是一个众所周知的问题,并且已经有了答案...请帮助并提供解决方案。
如果有任何库专门做这件事(并以单次飞行方式运行),那将非常有帮助。
谢谢
Can you help me? Let say we have:
- A service
S
that when you give an array of numberx
, it will give back an item detail for each id inside arrayx
. Assume this is an expensive call.
Now, you have a lot of request that come at the same time. If for each request brings an id, we can handle it with singleflight like the following code:
var group singleflight.Group
func GetResult(id int64) (itemDetail, error) {
key := fmt.Sprintf("Key%d",id) // key for identifier the same request id
result, err, _ := group.Do(key, func() (inteface{}, error) {
result, err := GetFromService(id) // Get data from the service S
if err!= nil {
return nil, err
}
return result, nil
}
return result, err
}
Now, what if for each request, there is an array of ids. Is there any way to handle all of the requests such that each id in x
will only queried to service S
once and the other duplicate ids will wait from something (if possible) to get the result?
For example, there are 3 concurrent in-flight request. The 1st request ids are 1,2,3
, the 2nd request ids are 1,2,4
, and the 3rd request ids are 1,3,4
. The target is to only call the service S
2 times. The 1st call will request for ids 1,2,3
. After this call is over, it can share the result to the 2nd and 3rd request and consequently, the 1st request is finished.
While the 1st call is still calling to the Service S, the 2nd request came and from there we know that this 2nd request only need to request for id 4, and thus subsequently do that (because the 1st call when finish will "share" the result to this 2nd call).
While the 1st call and 2nd call is in-flight, let's assume that the 3rd call came, and we know for sure that in the 3rd call.. it doesn't need to do anything other than waiting for the 1st request and 2nd request to finish. and after the 1st request and 2nd request finish, they will subsequently share their call result from X
and ending the 3rd request
I seem to can't find any question related to this on so after checking on so many threads... If this is a well known questions, and already have the answers... please help and provide the solution.
If there's any library that specifically does this thing (and operate in singleflight manner), that would be really help.
Thank you
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论