内存映射文件导致物理内存不足
我有 2GB RAM 并运行内存密集型应用程序，并且进入低可用物理内存状态，并且系统不响应用户操作，例如打开任何应用程序或菜单调用等。
我如何触发或告诉系统将内存交换到页面文件和空闲物理内存？ 我使用的是Windows XP。
如果我在 4GB RAM 机器上运行相同的应用程序，情况并非如此，系统响应良好。在可用物理内存耗尽后，系统会自动交换到页面文件并释放物理内存，这并不像 2GB 系统那么糟糕。
为了克服这个问题（在 2GB 机器上），尝试对由应用程序分配的大型数据集使用内存映射文件。在这种情况下，应用程序（进程）的虚拟内存很好，但系统缓存很高，并且与上面相同的问题是物理内存较少。
即使内存映射文件没有映射到进程虚拟内存系统缓存也很高。为什么？？？！！！ :(
任何帮助表示赞赏。 谢谢。
I have a 2GB RAM and running a memory intensive application and going to low available physical memory state and system is not responding to user actions, like opening any application or menu invocation etc.
How do I trigger or tell the system to swap the memory to pagefile and free physical memory?
I'm using Windows XP.
If I run the same application on 4GB RAM machine it is not the case, system response is good. After getting choked of available physical memory system automatically swaps to pagefile and free physical memory, not that bad as 2GB system.
To overcome this problem (on 2GB machine) attempted to use memory mapped files for large dataset which are allocated by application. In this case virtual memory of the application(process) is fine but system cache is high and same problem as above that physical memory is less.
Even though memory mapped file is not mapped to process virtual memory system cache is high. why???!!! :(
Any help is appreciated.
Thanks.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
如果使用内存映射文件的数据访问模式是顺序的，则通过在打开基础文件时指定 FILE_FLAG_SEQUENTIAL_SCAN 标志，您可能会获得稍微更好的页面回收。如果您的数据模式以随机顺序访问映射文件，这将无济于事。
您应该考虑减小地图视图的大小。这就是所有内存实际消耗和缓存的地方。由于您似乎需要处理大于可用连续空闲物理内存的文件，因此您可能可以比虚拟内存页面交换器做得更好，因为您比虚拟内存页面交换器更了解如何使用内存。内存管理器可以。如果可能的话，尝试调整您的设计，以便您可以使用较小的视图操作大文件的部分内容。
即使您无法摆脱对整个基础文件范围进行完全随机访问的需要，根据需要拆除并重新创建视图以将视图移动到文件中要访问的部分仍然可能是有益的。下一步操作需要访问。如果您的数据访问模式在继续之前倾向于聚集在文件的各个部分周围，那么您将不需要经常移动视图。您将需要拆除并重新创建视图对象，但由于拆除视图也会释放与该视图关联的所有缓存页面，因此您可能会看到性能的净增益，因为较小的视图显着降低了性能内存压力和系统范围内的页面交换。尝试根据已安装的系统 RAM 的一部分设置视图的大小，并根据文件处理的需要移动视图。视图越大，移动它的需要就越少，但它消耗的 RAM 就越多，可能会影响系统响应能力。
If your data access pattern for using the memory mapped file is sequential, you might get slightly better page recycling by specifying the FILE_FLAG_SEQUENTIAL_SCAN flag when opening the underlying file. If your data pattern accesses the mapped file in random order, this won't help.
You should consider decreasing the size of your map view. That's where all the memory is actually consumed and cached. Since it appears that you need to handle files that are larger than available contiguous free physical memory, you can probably do a better job of memory management than the virtual memory page swapper since you know more about how you're using the memory than the virtual memory manager does. If at all possible, try to adjust your design so that you can operate on portions of the large file using a smaller view.
Even if you can't get rid of the need for full random access across the entire range of the underlying file, it might still be beneficial to tear down and recreate the view as needed to move the view to the section of the file that the next operation needs to access. If your data access patterns tend to cluster around parts of the file before moving on, then you won't need to move the view as often. You'll take a hit to tear down and recreate the view object, but since tearing down the view also releases all the cached pages associated with the view, it seems likely you'd see a net gain in performance because the smaller view significantly reduces memory pressure and page swapping system wide. Try setting the size of the view based on a portion of the installed system RAM and move the view around as needed by your file processing. The larger the view, the less you'll need to move it around, but the more RAM it will consume potentially impacting system responsiveness.
正如我认为您在帖子中暗示的那样，响应时间缓慢可能至少部分是由于操作系统将内存内容写入页面文件以便为物理内存中的其他进程腾出空间时系统的延迟造成的。
显而易见的解决方案（可能不切实际）是在应用程序中使用更少的内存。我认为这不是一个选择，或者至少不是一个简单的选择。另一种方法是尝试主动将数据刷新到磁盘，以持续保留可用的物理内存供其他应用程序运行。您可以使用 GlobalMemoryStatusEx 查找计算机上的总内存。 GetProcessMemoryInfo 将返回有关您自己的应用程序内存使用情况的当前信息。由于您说您正在使用内存映射文件，因此您可能还需要考虑这一点。例如，我相信从该 API 返回的
PageFileUsage信息不会包含有关您自己的内存映射文件的信息。如果您的应用程序正在监视使用情况，您可以使用 FlushViewOfFile 主动将数据从内存强制写入磁盘。我认为还有一个 API (EmptyWorkingSet) 试图将尽可能多的脏页写入磁盘，但这似乎很可能会严重损害您自己的应用程序的性能。不过，在您知道应用程序即将进入某种空闲状态的情况下，它可能很有用。
最后，另一个可能有用的 API 是 SetProcessWorkingSetSizeEx。您可以考虑使用此 API 来提示应用程序工作集大小的上限。这可能有助于为其他应用程序保留更多内存。
编辑：这是另一个明显的陈述，但我忘了之前提到过。它对您来说可能也不实用，但考虑到您遇到 32 位限制，听起来您可能做的最好的事情之一就是将应用程序构建为 64 位并在 64 位操作系统上运行它（并在机器上增加一点内存）。
As I think you are hinting in your post, the slow response time is probably at least partially due to delays in the system while the OS writes the contents of memory to the pagefile to make room for other processes in physical memory.
The obvious solution (and possibly not practical) is to use less memory in your application. I'll assume that is not an option or at least not a simple option. The alternative is to try to proactively flush data to disk to continually keep available physical memory for other applications to run. You can find the total memory on the machine with GlobalMemoryStatusEx. And GetProcessMemoryInfo will return current information about your own application's memory usage. Since you say you are using a memory mapped file, you may need to account for that in addition. For example, I believe the
PageFileUsageinformation returned from that API will not include information about your own memory mapped file.If your application is monitoring the usage, you may be able to use FlushViewOfFile to proactively force data to disk from memory. There is also an API (EmptyWorkingSet) that I think attempts to write as many dirty pages to disk as possible, but that seems like it would very likely hurt performance of your own application significantly. Although, it could be useful in a situation where you know your application is going into some kind of idle state.
And, finally, one other API that might be useful is SetProcessWorkingSetSizeEx. You might consider using this API to give a hint on an upper limit for your application's working set size. This might help preserve more memory for other applications.
Edit: This is another obvious statement, but I forgot to mention it earlier. It also may not be practical for you, but it sounds like one of the best things you might do considering that you are running into 32-bit limitations is to build your application as 64-bit and run it on a 64-bit OS (and throw a little bit more memory at the machine).
嗯，听起来你的程序需要超过 2GB 的工作集。
现代操作系统的设计目的是始终将大部分 RAM 用于某些用途，只保留相当少量的空闲内存，以便可以立即将其分发给需要更多内存的进程。其余的用于保存最近使用过的内存页和缓存的磁盘块；最近未使用的内容将被刷新回磁盘以补充可用页面池。简而言之，不应该有太多可用物理内存。
使用普通内存分配和内存映射文件之间的主要区别在于，当必须将数据调出内存时，数据的存储位置。它不一定对内存何时被调出有任何影响，并且对调出内存所需的时间也几乎没有影响。
您看到的真正问题可能不是可用物理内存太少，而是分页率太高。
我的建议是尝试减少程序所需的存储量，并看看是否可以增加引用的局部性以减少所需的分页量。
Well, it sounds like your program needs more than 2GB of working set.
Modern operating systems are designed to use most of the RAM for something at all times, only keeping a fairly small amount free so that it can be immediately handed out to processes that need more. The rest is used to hold memory pages and cached disk blocks that have been used recently; whatever hasn't been used recently is flushed back to disk to replenish the pool of free pages. In short, there isn't supposed to be much free physical memory.
The principle difference between using a normal memory allocation and memory mapped a files is where the data gets stored when it must be paged out of memory. It doesn't necessarily have any effect on when the memory will be paged out, and will have little effect on the time it takes to page it out.
The real problem you are seeing is probably not that you have too little free physical memory, but that the paging rate is too high.
My suggestion would be to attempt to reduce the amount of storage needed by your program, and see if you can increase the locality of reference to reduce the amount of paging needed.