模拟物理磁盘以加快执行速度?

发布于 2024-08-03 05:09:26 字数 423 浏览 8 评论 0原文

我们有一个命令行 exe,它从文本文件获取输入并生成输出文本文件。它用于复杂的工业模拟。

这个exe的源代码早已不复存在。现在很容易创建一个 .NET 包装器来控制该 exe 的执行并通过 Web 服务链接到外部应用程序。

不幸的是,一个新的要求是对这个黑盒模型进行优化。现在有多种方法可以执行黑盒优化,但它们都需要调用可执行文件数千(数百万?)次。显然,基于磁盘的文本文件的创建和解析是模拟过程的瓶颈。

无论如何,我可以欺骗这个可执行文件不写入物理磁盘吗?如果我们在 Unix 上,我想管道可以解决这个问题,但我们的部署服务器是 Windows Server 03。

我只是想到 ramdrive 可能会解决这个问题,但自 MS-DOS 6 以来我还没有使用过其中之一。商业产品值得一看吗?有没有人有任何其他想法来通过代码模拟物理驱动器?我们使用 .NET 3.5。

We have a command line exe that takes input from a text file and produces an output text file. It is used for complex industrial simulations.

The source code for this exe is long gone. Now it was easy enough to create a .NET wrapper which controls the execution of this exe and links in with an external app via a web service.

Unfortunetely a new requirement is to run optimization over this black box model. Now there are various methods to perform black box optimization but they all require calling the executable thousands (millions?) of times. Its obvious that the creation and parsing of disk based text files is the bottleneck of the simulation process.

Is there anyway I can trick this executable into not writing to a physical disk? If we were on Unix I suppose pipes would do the trick, but our deployment server is Windows Server 03.

It just occurred to me that a ramdrive might solve this problem, but I havent played with one of those since MS-DOS 6. Any commercial products worth looking at? Does anyone have any other ideas for emulating a physical drive through code? We are on .NET 3.5.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

貪欢 2024-08-10 05:09:26

如果您运行的是 Vista,则有一个商业 Ramdisk 产品可能适合您。如果您的系统已经使用了大部分内存,则可能需要 x64 版本,以确保您最终不会进行过多的页面交换。

另一种选择是花一点钱购买 15000 RPM 磁盘或 SSD(固态磁盘),尽管这会比 RAM 磁盘慢。

但从长远来看,对处理工具进行逆向工程并从头开始重写以避免瓶颈可能会更便宜。

If you're running on Vista, there's a commercial Ramdisk product that may suit. The x64 version may be needed if your system is already using most of its memory, to make sure you don't end up doing too much page swapping.

Another option is to spend a bit of cash on a 15000 RPM disk or a SSD (solid state disk), although that'll be slower than a RAM disk.

In the long term though it may be cheaper to reverse engineer the processing tool and rewrite it from scratch to avoid the bottlenecks.

花辞树 2024-08-10 05:09:26

在 Windows 中也可以进行管道输出 - 如果可执行文件仅生成标准输出的输出,则可以使用它。

如果它实际上写入文件:由于 Windows 缓存文件系统写入,我可以想象简单地写入,然后删除小文件几乎与 ramdisk 一样快。您是否真的尝试过以实际的速度运行程序,并删除运行后的所有输出?如果您密切关注 CPU 使用率和磁盘队列,您应该知道普通的旧磁盘缓存是否足够。

Pipeing output is possible in windows as well - if the executable generates output only to standard out, you can use that.

If it actually writes to a file: since windows caches filesystem writes, I could imagine that simply writing, then deleting small files is almost as fast as a ramdisk. Have you actually tried running the program at a realistic rate, deleting all output post-run? If you keep an eye on CPU usage and the disk queue you should get an idea if plain old disk caching suffices.

为你拒绝所有暧昧 2024-08-10 05:09:26

如果您在虚拟机中运行它,则对其虚拟磁盘进行缓存是主机操作系统的责任...这意味着,您可以在虚拟机内或多或少地运行 Windows 服务器,并获得额外的缓存层。这在环境中有效吗?

If you run it in a virtual machine, caching for its virtual disks is the responsibility of the host OS... which means, you can run your Windows server inside a VM on more or less anything, and get an extra layer of caching. Does that work in the environment?

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文