在twisted中取消一组HTTP请求

发布于 2024-09-26 02:58:03 字数 412 浏览 1 评论 0原文

我正在使用twisted.web.client.getPage 发出多个HTTP 请求，并且希望能够根据用户的请求取消其中一些请求。理想情况下，我想做类似的事情：

# Pseudocode, getPage doesn't work like this:
getPage(url1, "group1")
getPage(url2, "group1")
getPage(url3, "group1")
...

# Later on
reactor.cancel_all("group1")

也许我可以将所有 Deferreds 添加到 DeferredList，但是我有很多小请求，所以大多数请求无论如何都会在给定时间完成（另外，我不知道如果您可以将 Deferreds 添加到现有的 DeferredList）...是否有更惯用的解决方案？

原文

I'm making several HTTP requests with twisted.web.client.getPage, and would like to be able to cancel some of them at the user's request. Ideally I would like to do something like:

# Pseudocode, getPage doesn't work like this:
getPage(url1, "group1")
getPage(url2, "group1")
getPage(url3, "group1")
...

# Later on
reactor.cancel_all("group1")

Maybe I could add all the Deferreds to a DeferredList, but I have a lot of small requests, so most of the requests would be finished at a given time anyway (plus, I don't know if you can add Deferreds to a existing DeferredList)... Is there a more idiomatic solution?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

枯寂 2024-10-03 02:58:03

您正在描述两个不同的问题。首先，使用 getPage 发出的 HTTP 请求是否可以被取消？不，不能。其次，是否可以将操作分组在一起，以便可以同时取消它们。当然，这并不涉及任何非常特殊的事情：

def cancel(group):
    for job in group:
        job.cancel()

group = []
group.append(job1)
group.append(job2)
...
cancel(group)

这里 Twisted 没有什么特别的 - 这只是创建一个集合，然后对其进行操作。你不需要反应堆来帮忙什么的。您所需要的是一种取消单个操作的方法。 Twisted 的最新版本添加了 Deferred.cancel （因此，与 pyfunc 的答案中链接的旧帖子相反，Deferreds 现在确实有被取消的概念）。然而，要真正执行任何操作，创建 Deferreds 的每个 API（例如 getPage）都必须更新以执行相关的取消操作。从 Twisted 10.1 开始，getPage 尚未更新。

因此，您可以实现 getPage 的取消（请将其贡献给 Twisted！），或者您可以忘记实际取消 HTTP 请求，而只是在结果到达时忽略它。

You are describing two separate issues. First, can an HTTP request made with getPage be cancelled at all? No, it can't. Second, can operations be grouped together so that they can all be cancelled simultaneously. Sure, that doesn't involve anything very special:

def cancel(group):
    for job in group:
        job.cancel()

group = []
group.append(job1)
group.append(job2)
...
cancel(group)

Nothing particular about Twisted here - this is just creating a collection and then operating on it. You don't need the reactor to help or anything. What you do need is a way to cancel an individual operation. The latest release of Twisted adds Deferred.cancel (so, contrary to the older post linked to in pyfunc's answer, Deferreds do have a notion of being cancelled now). However, for this to actually do anything, each API which creates Deferreds - for example, getPage - has to be updated to perform the relevant cancellation operation. As of Twisted 10.1, getPage has not been updated.

So you can either implement cancellation for getPage (and contribute it to Twisted, please!) or you can forget about actually cancelling the HTTP request and instead just ignore the result when it arrives.

回复收藏 0 原文