了解 *nix 上异步编程的基础知识

发布于 2024-12-25 10:06:57 字数 987 浏览 0 评论 0原文

一段时间以来,我一直在谷歌上进行大量搜索,以了解在 nix 机器上实现异步编程/行为的各种方法,并且(正如我之前所知)得到确认,仍然没有真正的异步模式(并发) Linux 上使用单线程),Windows 上也可以使用(IOCP)。

以下是针对 Linux 的几种替代方案:

  1. select/poll/epoll :: 无法使用单线程完成,因为 epoll 仍然阻塞调用。此外,受监视的文件描述符必须以非阻塞模式打开。
  2. libaio:: 我所知道的是,它的实现很糟糕,而且它仍然基于通知,而不是像 Windows I/O 完成端口那样基于完成。
  3. Boost ASIO :: 它在 Linux 下使用 epoll,因此不是真正的异步模式,因为它生成完全从用户代码中抽象出来的线程以实现前摄器设计模式
  4. libevent ::如果我更喜欢 ASIO,有什么理由去选择它?

现在问题来了:)

  1. 使用 epoll 编写快速可扩展网络服务器的最佳设计模式是什么(当然,必须在这里使用线程:()
  2. 我在某处读到“只有套接字可以在非阻塞模式下打开” " 因此 epoll 仅支持套接字,因此不能用于磁盘 I/O。 上面的说法有多真实?为什么不能使用 epoll 在磁盘 I/O 上进行异步编程?
  3. Boost ASIO 在 epoll 调用周围使用了一把大锁。我实际上并不理解它的含义以及如何使用 asio 本身来克服它。 类似问题
  4. 如何修改 ASIO 模式以处理磁盘文件?有推荐的设计模式吗?

希望有人能够用很好的解释来回答所有问题。任何解释 epoll 和 AIO 设计模式的实现细节的源链接也值得赞赏。

For some time now I have been googling a lot to get to know about the various ways to acheive asynchronous programming/behavior on nix machines and ( as known earlier to me ) got confirmed on the fact that there is still no TRULY async pattern (concurrency using single thread) for Linux as available for Windows(IOCP).

Below are the few alternatives present for linux:

  1. select/poll/epoll :: Cannot be done using single thread as epoll is still blocking call. Also the monitored file descriptors must be opened in non-blocking mode.
  2. libaio:: What I have come to know about is that its implementation sucks and its still notification based instead of being completion based as with windows I/O completion ports.
  3. Boost ASIO :: It uses epoll under linux and thus not a true async pattern as it spawns thread which are completely abstracted from user code to acheive the proactor design pattern
  4. libevent :: Any reason to go for it if I prefer ASIO?

Now Here comes the questions :)

  1. What would be the best design pattern for writing fast scalable network server using epoll (ofcourse, will have to use threads here :( )
  2. I had read somewhere that "only sockets can be opened in non-blocking mode" hence epoll supports only sockets and hence cannot be used for disk I/O.
    How true is the above statement and why async programming cannot be done on disk I/O using epoll ?
  3. Boost ASIO uses one big lock around epoll call. I didnt actually understand what can be its implications and how to overcome it using asio itself. Similar question
  4. How can I modify ASIO pattern to work with disk files? Is there any recommended design pattern ?

Hope somebody will able to answer all the questions with nice explanations also. Any link to source where the implementation details of epoll and AIO design patterns are exaplained is also appreciated.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

怎樣才叫好 2025-01-01 10:06:57

Boost ASIO :: 它在Linux下使用epoll,因此不是真正的异步
模式,因为它产生完全从用户抽象的线程
实现前摄器设计模式的代码

这是不正确的。 Asio 库在最新的 Linux 内核版本上默认使用 epoll()。但是,调用 io_service::run() 的线程将根据需要调用回调处理程序。 Asio 库中只有一个地方使用线程来模拟异步接口,文档

每个 io_service 都有一个额外的线程用于模拟异步
主机分辨率。该线程是在第一次调用时创建的
ip::tcp::resolver::async_resolve()
ip::udp::resolver::async_resolve()

这并不会使该库“不是真正的异步模式”,正如您声称的那样,事实上,根据定义,它的名称与您不一致。

1) 使用 epoll 编写快速可扩展网络服务器的最佳设计模式是什么(当然,这里必须使用线程:(

我建议使用Boost Asio,它使用proactor设计模式。

3) Boost ASIO 在 epoll 调用周围使用一大锁。我实际上并没有
了解它的含义以及如何克服它
asio本身

epoll 反应器使用互斥体来调度处理程序,尽管实际上这对于大多数应用程序来说并不是一个大问题。有一些特定于应用程序的方法可以缓解这种行为,例如每个 CPU 的 io_service 来利用数据局部性。请参阅我对有关此主题的类似问题的回答Asio 邮件列表也经常对此进行讨论。

4) 如何修改 ASIO 模式以处理磁盘文件?有没有
推荐的设计模式?

正如您所指出的,Asio 库本身并不支持文件 I/O。已经多次尝试将其添加到库中,我建议在

Boost ASIO :: It uses epoll under linux and thus not a true async
pattern as it spawns thread which are completely abstracted from user
code to acheive the proactor design pattern

This is not correct. The Asio library uses epoll() by default on most recent Linux kernel versions. however, threads invoking io_service::run() will invoke callback handlers as needed. There is only one place in the Asio library that a thread is used to emulate an asynchronous interface, it is well described in the documentation:

An additional thread per io_service is used to emulate asynchronous
host resolution. This thread is created on the first call to either
ip::tcp::resolver::async_resolve() or
ip::udp::resolver::async_resolve().

This does not make the library "not a true async pattern" as you claim, in fact its name would disagree with you by definition.

1) What would be the best design pattern for writing fast scalable network server using epoll (of course, will have to use threads here :(
)

I suggest using Boost Asio, it uses the proactor design pattern.

3) Boost ASIO uses one big lock around epoll call. I didnt actually
understand what can be its implications and how to overcome it using
asio itself

The epoll reactor uses a mutex to dispatch handlers, though in practice this is not a big concern for most applications. There are application specific ways to mitigate this behavior, such as an io_service per CPU to exploit data locality. See my answer to a similar question on this topic. It is also discussed on the Asio mailing list frequently.

4) How can I modify ASIO pattern to work with disk files? Is there any
recommended design pattern?

The Asio library does not natively support file I/O as you noted. There have been several attempts to add it to the library, I'd suggest discussing on the mailing list.

咿呀咿呀哟 2025-01-01 10:06:57

首先:

得到确认,Linux 仍然没有真正的异步模式(使用单线程的并发),可用于 Windows (IOCP)。

您可能有一个小误解,异步可以构建在“轮询”api 之上。

更重要的是,“reactor”(类似 epoll)API 比“proactor”API (IOCP) 更强大,因为
第二个可以根据第一个来实现(但反之则不然)。

还有一些“真正”异步的操作,例如磁盘 I/O,一些其他工具可以与信号组合,并且 Linux 特定的 signalfd 可以提供其他一些情况的完整覆盖。

底线。 epoll 是真正的异步 I/O

First of all:

got confirmed on the fact that there is still no TRULY async pattern (concurrency using single thread) for Linux as available for Windows(IOCP).

You probably has a small misconception, asynchronous can be build on top of "polling" api.

More then that "reactor" (epoll-like) API is more powerful then "proactor" API (IOCP) as
the second can be implemented in terms of the first one (but not the other way around).

Also some operations that are "truly" asynchronous for example like disk I/O, some some other tools can be with combination of signals and Linux specific signalfd can provide full coverage of some other cases.

Bottom line. epoll is truly asynchronous I/O

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文