posix 读取延迟()

发布于 2024-10-21 21:38:58 字数 335 浏览 4 评论 0原文

我们有一个对延迟非常敏感的应用程序,从某种意义上说,读取时的延迟峰值非常非常糟糕。

我已经测试了 XFS 和 ext4,将 O_ASYNC 写入文件,然后最后使用 fdatasync() 可能会导致读取延迟达到 1 秒或更长时间!

然后我尝试了 O_SYNC,读取延迟更加稳定,但写入文件的速度非常慢。

因此,我尝试编写 O_ASYNC 并同步写入文件的每 5 MB,它的速度很快,读取延迟也相当稳定。

然而,30分钟我仍然可以读到需要一秒或更长时间的内容。

如果您在 Linux 上构建了对延迟敏感的应用程序,您如何处理文件系统,或者您根本不使用它并将设备作为原始设备安装?

we have a very latency sensitive application, in the sense that latency spikes on reads are very very bad.

I’ve tested XFS and ext4, and writing O_ASYNC to the file and then fdatasync() at the end can lead to 1 second or more spikes in read latency!

i then tried O_SYNC and I got much more stable read latencies but writing to the file is extremely slow.

So, I tried writing O_ASYNC and syncing every 5 megabytes written to the file, and its fast and the read latencies are reasonably stable as well.

However, 30 minutes I can still get a read that takes a second or more.

If you built latency sensitive applications on Linux, how did YOU deal with working with the filesystem, or did you just not use it at all and mount the device as a RAW device?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

泪意 2024-10-28 21:38:58

文件系统总是会增加少量延迟,因此对于真正对延迟敏感的应用程序,我会考虑通过使用原始设备或绕过操作系统缓存使用 O_DIRECT 打开文件来绕过文件系统。

SSD 存储延迟的其他技巧包括:

  • 通过从多个线程读取/写入来利用现代 SSD 中固有的并行性。延迟不会减少,但如果真正的问题是 IOPS,这会有所帮助。
  • 检查 SSD 上文件系统的对齐情况。如果做得不好,性能会减半,延迟会加倍。
  • 检查磁盘调度算法。 Linux“noop”调度程序通常最适合基于 SSD 的存储。
  • 使用更好的 SSD 硬件,例如基于 PCIe 的 SSD,通常可提供更低的延迟

也就是说:一秒的读取时间听起来并不正确。要么负载/利用率太高,以至于硬件根本无法应对,更多/更好的硬件将是最好的解决方案,要么是其他严重错误。

A filesystem always adds a small amount of latency, so for a really latency sensitive application I would consider bypassing the file system by using the raw device or opening the file with O_DIRECT bypassing the OS caches.

Other tricks for latency on SSD storage are:

  • use the inherent parallelism in modern SSDs by reading/writing from multiple threads. The latency will not decrease, but if the real issue are IOPS this helps.
  • Check the alignment of the filesystem on the SSD. If this is not done right, you half the performance and double the latency.
  • Check the disk scheduling algorithms. Linux "noop" scheduler is usually the best for SSD based storage.
  • Use better SSD hardware, e.g. PCIe based SSDs that usually provide a much lower latency

That said: A read time from a second simply doesn't sound right. Either the load/utilization is so high that the hardware simply can't cope with it and more/better hardware would be the best solution or something else is terribly wrong.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文