我应该对转换和删除的图片使用虚拟磁盘吗?

发布于 2024-12-18 07:39:05 字数 682 浏览 1 评论 0原文

我这里有一个小程序(python 2.7),它在一台旧机器上运行,它基本上通过运行外部二进制文件不断获取图片(用于延时),并将它们转换为有效的格式以节省磁盘空间。

我想尽量减少磁盘操作,因为它已经很旧了,我希望它能持续更长时间。

此时,程序将相机中的数据写入磁盘,然后对其进行转换并删除原始数据。然而,它对每个图像都会这样做,1-在磁盘上写入一个大文件,2-读取它进行转换,3-然后删除它......一堆不必要的磁盘操作,可以在内存中完成,因为原始文件不必存储,仅用作创建另一个文件的基础。

我确信 ramdisk 是解决方案,然后我在 google 上搜索了如何做到这一点,谷歌给我返回了一堆不鼓励使用 ramdisk 的链接,原因有很多:因为它们在现代系统中没有用(我是运行一个相当新的 Linux 内核);仅当您想要解密不应写入磁盘的数据时才应使用它们;一些测试表明 ramdisk 实际上可能比 hd 慢;操作系统有缓存...

所以我很困惑...

在这种情况下,我应该使用ramdisk吗?

谢谢。

PS:如果您想了解更多信息:我有一个专有的高分辨率相机,以及一个用于捕获单个图像的专有二进制文件,我可以指定它将写入文件的位置,这是一个巨大的 TIFF 文件,然后python程序运行imagemagick中的convert程序将其转换为JPEG,然后将其压缩为tar.bz2,因此质量几乎相同,但文件大小是TIFF的1/50。

I have a little program here (python 2.7) that runs on an old machine and it basically keeps getting pictures (for timelapses) by running an external binary and converts them to an efficient format to save up disk space.

I want to minimize the disk operations, because it's already pretty old and I want it to last some more time.

At the moment the program writes the data from the camera on the disk, then converts it and removes the original data. However it does that for every image, 1- it writes a large file on disk, 2- reads it to convert, 3- and then deletes it... a bunch of disc operations that aren't necessary and could be done in ram, because the original file doesn't have to be stored and is only used as a basis to create another one.

I was sure a ramdisk was the solution, then I googled on how to do that, and google returned me a bunch of links that discourage the use of ramdisk, the reasons are many: because they are not useful in modern systems (i'm running a pretty new linux kernel); they should only be used if you want to decrypt data that shouldn't hit the disk; some tests shows that ramdisk could be actually slower than hd; the operating system has a cache...

So I'm confused...

In this situation, should I use a ramdisk?

Thank you.

PS: If you want more info: I have a proprietary high-res camera, and a proprietary binary that I run to capture a single image, I can specify where it will write the file, which is a huge TIFF file, and then the python program runs the convert program from imagemagick to convert it to JPEG and then compress it in tar.bz2, so the quality is almost the same but the filesize is 1/50 of the TIFF.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

︶ ̄淡然 2024-12-25 07:39:05

我对 ramdisk 的体验与您在这里提到的一致。当我转向它们时,我失去了性能,因为可供内核智能缓存的内存较少,这把事情搞砸了。

但是,从您的问题来看,我了解到您想要优化磁盘操作的数量而不是速度,在这种情况下 RAM 磁盘可能有意义。与大多数此类问题一样,监控是正确的方法。

另一件让我印象深刻的事情是,如果您的原始映像没有那么大,您可能需要购买一个便宜的 USB 记忆棒并在其上而不是在主驱动器上执行 I/O。难道这不是一个选择吗?

My experience with ramdisks is congruent with what you've mentioned here. I lost performance when I moved to them because there was less memory available for the kernel to do it's caching intelligently and that messed things up.

However, from your question, I understand that you want to optimise for number of disk operations rather than speed in which case a RAM disk might make sense. As with most of these kinds of problems, monitoring is the right way to do it.

Another thing that struck me was that if your original image is not that big, you might want to buy a cheap USB stick and do the I/O on that rather than on your main drive. Is that not an option?

浪荡不羁 2024-12-25 07:39:05

啊,专有的二进制文件只提供某些选项。耶。最简单的解决方案是添加固态硬盘。您仍将保存到磁盘,但磁盘 IO 对于读取和写入来说会更高。

更好的解决方案是将 tiff 输出到 stdout(可能以不同的格式),并将其通过管道传输到 python 程序。它根本不会影响硬盘,但会增加工作量。当然,如果二进制文件不允许您这样做,那么它就没有意义了。

Ah, proprietary binaries that only give certain options. Yay. The simplest solution would be adding a solid state hard drive. You will still be saving to disk, but disk IO will be much higher for reading and writing.

A better solution would be outputting the tiff to stdout, perhaps in a different format, and piping it to your python program. It would never hit the hard drive at all, but it would be more work. Of course, if the binary doesn't allow you to do this, then it's moot.

GRAY°灰色天空 2024-12-25 07:39:05

如果在 Debian(可能还有其衍生版本)上,请使用“/run/shm”目录。

If on Debian (and possibly its derivatives), use "/run/shm" directory.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文