模拟物理磁盘以加快执行速度?
我们有一个命令行 exe,它从文本文件获取输入并生成输出文本文件。它用于复杂的工业模拟。
这个exe的源代码早已不复存在。现在很容易创建一个 .NET 包装器来控制该 exe 的执行并通过 Web 服务链接到外部应用程序。
不幸的是,一个新的要求是对这个黑盒模型进行优化。现在有多种方法可以执行黑盒优化,但它们都需要调用可执行文件数千(数百万?)次。显然,基于磁盘的文本文件的创建和解析是模拟过程的瓶颈。
无论如何,我可以欺骗这个可执行文件不写入物理磁盘吗?如果我们在 Unix 上,我想管道可以解决这个问题,但我们的部署服务器是 Windows Server 03。
我只是想到 ramdrive 可能会解决这个问题,但自 MS-DOS 6 以来我还没有使用过其中之一。商业产品值得一看吗?有没有人有任何其他想法来通过代码模拟物理驱动器?我们使用 .NET 3.5。
We have a command line exe that takes input from a text file and produces an output text file. It is used for complex industrial simulations.
The source code for this exe is long gone. Now it was easy enough to create a .NET wrapper which controls the execution of this exe and links in with an external app via a web service.
Unfortunetely a new requirement is to run optimization over this black box model. Now there are various methods to perform black box optimization but they all require calling the executable thousands (millions?) of times. Its obvious that the creation and parsing of disk based text files is the bottleneck of the simulation process.
Is there anyway I can trick this executable into not writing to a physical disk? If we were on Unix I suppose pipes would do the trick, but our deployment server is Windows Server 03.
It just occurred to me that a ramdrive might solve this problem, but I havent played with one of those since MS-DOS 6. Any commercial products worth looking at? Does anyone have any other ideas for emulating a physical drive through code? We are on .NET 3.5.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
免费软件版本的 RAMDisk
微软版本
操作方法
RAMDisk with freeware version
Microsoft version
How-to
如果您运行的是 Vista,则有一个商业 Ramdisk 产品可能适合您。如果您的系统已经使用了大部分内存,则可能需要 x64 版本,以确保您最终不会进行过多的页面交换。
另一种选择是花一点钱购买 15000 RPM 磁盘或 SSD(固态磁盘),尽管这会比 RAM 磁盘慢。
但从长远来看,对处理工具进行逆向工程并从头开始重写以避免瓶颈可能会更便宜。
If you're running on Vista, there's a commercial Ramdisk product that may suit. The x64 version may be needed if your system is already using most of its memory, to make sure you don't end up doing too much page swapping.
Another option is to spend a bit of cash on a 15000 RPM disk or a SSD (solid state disk), although that'll be slower than a RAM disk.
In the long term though it may be cheaper to reverse engineer the processing tool and rewrite it from scratch to avoid the bottlenecks.
在 Windows 中也可以进行管道输出 - 如果可执行文件仅生成标准输出的输出,则可以使用它。
如果它实际上写入文件:由于 Windows 缓存文件系统写入,我可以想象简单地写入,然后删除小文件几乎与 ramdisk 一样快。您是否真的尝试过以实际的速度运行程序,并删除运行后的所有输出?如果您密切关注 CPU 使用率和磁盘队列,您应该知道普通的旧磁盘缓存是否足够。
Pipeing output is possible in windows as well - if the executable generates output only to standard out, you can use that.
If it actually writes to a file: since windows caches filesystem writes, I could imagine that simply writing, then deleting small files is almost as fast as a ramdisk. Have you actually tried running the program at a realistic rate, deleting all output post-run? If you keep an eye on CPU usage and the disk queue you should get an idea if plain old disk caching suffices.
如果您在虚拟机中运行它,则对其虚拟磁盘进行缓存是主机操作系统的责任...这意味着,您可以在虚拟机内或多或少地运行 Windows 服务器,并获得额外的缓存层。这在环境中有效吗?
If you run it in a virtual machine, caching for its virtual disks is the responsibility of the host OS... which means, you can run your Windows server inside a VM on more or less anything, and get an extra layer of caching. Does that work in the environment?