多线程 Libcur
我需要使用 Libcurl 执行并行 HTTP 请求。
据我了解,我需要为每个线程创建一个新句柄 并将 CURLOPT_WRITEDATA
与某种线程本地存储一起使用。 多接口是否使这项任务变得更容易一些?
我也在使用 cookie,使用 CURLOPT_COOKIEFILE
和 CURLOPT_COOKIEJAR
会使 Libcurl为每个线程加载cookie文件?
I need to execute parallel HTTP requests using Libcurl.
From what I understand I need to create a new handle for each thread
and use CURLOPT_WRITEDATA
with some kind of Thread Local Storage.
Does the multi interface makes this task a little easier?
I'm also using cookies, does using CURLOPT_COOKIEFILE
and CURLOPT_COOKIEJAR
will make
Libcurl load the cookie file for each thread?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
您可能知道,libcurl 不是线程安全的,因此您应该确保 libcurl 句柄永远不会在多个线程之间共享。每个线程都需要存储本地数据(其中包括连接句柄)。
由此可见,对于每个句柄,即对于每个线程,libcurl 将需要从头开始读取 cookie 文件,因为该信息无法共享。在我看来,这不是问题,尽管更新它时可能会出现问题(您将有多个线程尝试它)。
关于多接口,它允许您复用多个传输,因此这是您尝试执行的操作的另一种方法,但在单个线程中。
2013 年 3 月更新
libcurl 现在是线程安全的。
As you probably know, libcurl is not thread safe, so you should ensure that the libcurl handle is never shared between multiple threads. Each thread will need to store local data (among other things, the connection handle).
From this, it ensues that for each handle, i.e., for each thread, libcurl will need to read the cookie file from scratch, since this information cannot be shared. This is not a problem, in my opinion, although there could be issues when updating it (you will have multiple thread attempting it).
About the multi interface, it allows you to multiplex multiple transfers, so it is another approach to what you are trying to do but in a single thread.
UPDATE March 2013
libcurl is now thread-safe.
这不是一个直接的答案,但是为什么并行 HTTP 请求需要多线程呢?
多接口就是为此目的而设计的:您添加多个句柄,然后通过一次调用处理所有这些句柄,所有操作都在同一线程中。来自文档:
如果您想要多个线程,我建议您在每个线程中使用简单接口,而忘记多接口。
共享只是在简单手柄之间共享数据,您可以使用带/不带多接口的接口。如果您确实使用多个线程,则必须提供自己的锁定。
This is not a direct answer, but why do you need multithreading for parallel HTTP requests?
The multi interface is designed for this purpose: you add multiple handles and then process all of them with one call, all in the same thread. From the documentation:
If you want multiple threads, I suggest you use the easy interface in each thread, and forget about the multi interface.
Sharing simply shares data between easy handles, you can use the interface with/without the multi interface. If you do use multiple threads, you have to provide your own locking.
另请查看 libcurl 共享接口。它是为此目的而设计的,即在请求之间共享数据:
Also check out libcurl share interface. It was designed for this purpose, i.e. to share data between requests: