osx 上的 aio:它是在内核中实现还是通过用户线程实现?其他选择?

发布于 2024-11-05 01:26:23 字数 490 浏览 1 评论 0原文

我正在开发我的小型 C++ 框架,并且有一个文件类,它也应该支持异步读写。除了在我发现的一些工作线程中使用同步文件 I/O 之外,唯一的解决方案是 aio。无论如何,我环顾四周并在某处读到,在 Linux 中,aio 甚至不是在内核中实现的,而是通过用户线程实现的。 OSX 也是如此吗?另一个问题是 aio 的回调功能,它必须为每个回调生成一个额外的线程,因为您无法分配某个线程或线程池来处理这个问题(信号对我来说不是一个选项)。因此,以下是由此产生的问题:

  • aio 是否在 osx 内核中实现,因此很可能比我自己的线程实现更好?

  • 回调系统 - 为每个回调生成一个线程 - 会成为实践中的瓶颈吗?

  • 如果 aio 不值得在 osx 上使用,那么 unix 上还有其他替代方案吗?在可可中?在carbon 中?

  • 或者我应该简单地使用我自己的线程池模拟异步 I/O?

您在这个问题上有什么经验?

I am working on my small c++ framework and have a file class which should also support async reading and writing. The only solution other than using synchronous file i/o inside some worker threads I found is aio. Anyways I was looking around and read somewhere, that in Linux, aio is not even implemented in the kernel but rather with user threads. Is the same true for OSX? Another concern is aio's functionality of callbacks which has to spawn an extra thread for each callback since you can't assign a certain thread or threadpool to take care of that (signals are not an option for me). So here are the questions resulting from that:

  • Is aio implemented in the Kernel of osx and thus is most likely better than my own threaded implementation?

  • Can the callback system -spawning a thread for each callback- become a bottleneck in practice?

  • If aio is not worth using on osx, are there any other alternatives on unix? in cocoa? in carbon?

  • Or should I simply emulate async i/o with my own threadpool?

What is your experience on the subject?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

-柠檬树下少年和吉他 2024-11-12 01:26:23

您可以准确地看到 AIO 在 OSX 上的实现方式 这里

该实现使用内核线程,这是一个作业队列,每个线程根据每个请求的优先级在优先级队列中弹出并以阻塞方式执行(至少乍一看是这样的)。

您可以使用 sysctl 配置线程数和队列大小。要查看这些选项和默认值,请运行 sysctl -a | grep aio

kern.aiomax = 90
kern.aioprocmax = 16
kern.aiothreads = 4

根据我的经验,为了使使用 AIO 有意义,这些限制需要高得多。

至于线程中的回调,我不认为 Mac OS X 支持。它仅通过信号进行完成通知(请参阅源代码)。

您可能可以在自己的线程池中完成同样好的工作。您可以比当前 darwin 实现做得更好的一件事是按磁盘上的物理位置对读取作业进行排序(请参阅 fcntl 和 F_LOG2PHYS ),这甚至可能会给您带来优势。

You can see exactly how AIO is implemented on OSX right here.

The implementation uses kernel threads, one queue of jobs which each thread pops and execute in a blocking fashion in a priority queue based on each request's priority (at least that's what it looks like at a first glance).

You can configure the number of threads and the size of the queue with sysctl. To see these options and the default values, run sysctl -a | grep aio

kern.aiomax = 90
kern.aioprocmax = 16
kern.aiothreads = 4

In my experience, in order for it to make any sense to use AIO, these limits need to be a lot higher.

As for the callbacks in threads, I don't believe Mac OS X supports that. It only does completion notifications through signals (see source).

You could probably do as good of a job in your own thread pool. One thing you could do better than the current darwin implementation is to sort your read jobs by physical location on the disk (see fcntl and F_LOG2PHYS) which might even give you an edge.

诺曦 2024-11-12 01:26:23

@Moka:抱歉,你对linux实现的理解是错误的,从内核2.6开始,有一个AIO的内核实现,它来自于libaio (libaio.h),

但没有实现POSIX.1 AIO 使用内核线程,而不是使用用户线程,这样做是为了使其更具可移植性,因为并非所有基于 UNIX 的操作系统都支持内核级别的完成事件。

@Moka: Sorry to say that you're wrong on the linux implementation, as of kernel 2.6 there is a kernel implementation of AIO, which comes in libaio (libaio.h)

The implementation that doesn't use Kernel threads but instead uses user threads is POSIX.1 AIO, and it does it that way to make it more portable, as not all unix based OS support completion events at Kernel level.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文