防止繁重的进程陷入交换文件中

发布于 2024-07-19 00:28:55 字数 519 浏览 10 评论 0原文

我们的服务往往会在晚上在客户的服务器上睡着,然后很难醒来。 似乎发生的情况是,进程堆(有时有数百 MB)被移动到交换文件中。 这种情况发生在晚上,当我们的服务未使用时,其他服务计划运行(数据库备份、AV 扫描等)。 发生这种情况时,在几个小时不活动后,对服务的第一次调用最多需要几分钟(后续调用需要几秒钟)。

我很确定这是虚拟内存管理的问题,而且我真的很讨厌强制操作系统将我们的服务保留在物理内存中的想法。 我知道这样做会损害服务器上的其他进程,并降低服务器的整体吞吐量。 话虽如此,我们的客户只是希望我们的应用程序能够响应。 他们不在乎夜间工作是否需要更长的时间。

我依稀记得有一种方法可以强制 Windows 将页面保留在物理内存上,但我真的很讨厌这个想法。 我更倾向于一些内部或外部看门狗,它们将启动更高级别的功能(已经有一些内部调度程序做得很少,并且没有区别)。 如果有一个第三方工具提供这种服务那就太好了。

我很想听到任何评论、建议和此类问题的常见解决方案。 该服务采用VC2005编写,运行在Windows服务器上。

Our service tends to fall asleep during the nights on our client's server, and then have a hard time waking up. What seems to happen is that the process heap, which is sometimes several hundreds of MB, is moved to the swap file. This happens at night, when our service is not used, and others are scheduled to run (DB backups, AV scans etc). When this happens, after a few hours of inactivity the first call to the service takes up to a few minutes (consequent calls take seconds).

I'm quite certain it's an issue of virtual memory management, and I really hate the idea of forcing the OS to keep our service in the physical memory. I know doing that will hurt other processes on the server, and decrease the overall server throughput. Having that said, our clients just want our app to be responsive. They don't care if nightly jobs take longer.

I vaguely remember there's a way to force Windows to keep pages on the physical memory, but I really hate that idea. I'm leaning more towards some internal or external watchdog that will initiate higher-level functionalities (there is already some internal scheduler that does very little, and makes no difference). If there were a 3rd party tool that provided that kind of service is would have been just as good.

I'd love to hear any comments, recommendations and common solutions to this kind of problem. The service is written in VC2005 and runs on Windows servers.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

吃颗糖壮壮胆 2024-07-26 00:28:55

正如您所提到的,强制应用程序保留在内存中并不是在计算机上共享资源的最佳方式。 您可能会发现效果良好的快速解决方案是简单地安排一个事件,在每天早上的特定时间唤醒您的服务,然后您的客户开始使用它。 您只需使用简单的脚本或 EXE 调用在 Windows 任务计划程序中安排它即可。

As you mentioned, forcing the app to stay in memory isn't the best way to share resources on the machine. A quick solution that you might find that works well is to simply schedule an event that wakes your service up at a specific time each morning before your clients start to use it. You can just schedule it in the windows task scheduler with a simple script or EXE call.

凑诗 2024-07-26 00:28:55

我并不是说您想要这样做,或者这是最佳实践,但您可能会发现它对您来说足够有效。 它似乎符合您的要求。

摘要:定期触摸流程中的每一页,一次一页。

一个在后台运行并每 N 秒唤醒一次的线程怎么样? 每次页面唤醒时,它都会尝试从地址 X 读取。该尝试受到异常处理程序的保护,以防您读取错误地址。 然后将 X 添加页面的大小。

4GB 中有 65536 个页面,3GB 中有 49152 个页面,2GB 中有 32768 个页面。 将您的空闲时间(夜间停滞时间)除以您想要(尝试)访问每个页面的频率。

BYTE *ptr;

ptr = NULL;
while(TRUE)
{
    __try
    {
        BYTE b;

        b = *ptr;
    }
    __except(EXCEPTION_EXECUTE_HANDLER)
    {
        // ignore, some pages won't be accessible
    }

    ptr += sizeofVMPage;

    Sleep(N * 1000);
}

您可以从 GetSystemInfo() 返回结果中的 dwPageSize 值获取 sizeOfVMPage 值。

不要尝试使用 if (!IsBadReadPtr(ptr)) 来避免异常处理程序,因为应用程序中的其他线程可能会同时修改内存保护。 如果你因此而陷入困境,几乎不可能找出原因(这很可能是不可重复的竞争条件),所以不要浪费时间。

当然,您希望在白天关闭该线程,并且仅在死区时间运行它。

I'm not saying you want to do this, or that it is best practice, but you may find it works well enough for you. It seems to match what you've asked for.

Summary: Touch every page in the process, on page at a time, on a regular basis.

What about a thread that runs in the background and wakes up once every N seconds. Each time the page wakes up, it attempts to read from address X. The attempt is protected with an exception handler in case you read a bad address. Then increment X by the size of a page.

There are 65536 pages in 4GB, 49152 pages in 3GB, 32768 pages in 2GB. Divide your idle time (overnight dead time) by how often you want (attempt) to hit each page.

BYTE *ptr;

ptr = NULL;
while(TRUE)
{
    __try
    {
        BYTE b;

        b = *ptr;
    }
    __except(EXCEPTION_EXECUTE_HANDLER)
    {
        // ignore, some pages won't be accessible
    }

    ptr += sizeofVMPage;

    Sleep(N * 1000);
}

You can get the sizeOfVMPage value from the dwPageSize value in the returned result from GetSystemInfo().

Don't try to avoid the exception handler by using if (!IsBadReadPtr(ptr)) because other threads in the app may be modifying memory protections at the same time. If you get unstuck because of this it will almost impossible to identify why (it will most likely be a non-repeatable race condition), so don't waste time with it.

Of course, you'd want to turn this thread off during the day and only run it during your dead-time.

恏ㄋ傷疤忘ㄋ疼 2024-07-26 00:28:55

第三种方法可能是让您的服务运行一个线程,该线程执行一些琐碎的事情,例如递增计数器,然后休眠相当长的一段时间,例如 10 秒。 Thios 对其他应用程序的影响应该很小,但至少保留一些页面可用。

A third approach could be to have your service run a thread that does something trivial like incrementing a counter and then sleeps for a fairly long period, say 10 seconds. Thios should have minimal effect on other applications but keep at least some of your pages available.

终陌 2024-07-26 00:28:55

另一件要确保的事情是您的数据是本地化的。

换句话说:您真的需要全部 300 MiB 内存才能执行任何操作吗? 您使用的数据结构是否可以重新排列,以便仅用几兆字节即可满足任何特定请求?

例如,

  • 如果您的 300 MiB 堆内存包含面部识别数据。 内部可以安排数据,让男女人脸数据存储在一起吗? 或者大鼻子和小鼻子是分开的?

  • 如果它有某种逻辑结构,它可以排序吗? 这样二分搜索就可以用来跳过很多页面?

  • 如果它是一个专有的内存数据库引擎,是否可以更好地对数据进行索引/集群,以便不需要如此多的内存页面点击?

  • 如果它们是图像纹理,常用纹理可以彼此靠近吗?

您真的需要全部 300 MiB 内存才能执行任何操作吗? 如果没有所有数据返回到内存中,您就无法服务请求吗?

其他:安排任务在 6 点唤醒它。

The other thing to ensure is that your data is localized.

In other words: do you really need all 300 MiB of the memory before you can do anything? Can the data structures you use be rearranged so that any particular request could be satisfied with only a few megabytes?

For example

  • if your 300 MiB of heap memory contains facial recognition data. Can the data internally be arranged so that male and female face data are stored together? Or big-noes are separate from small-noses?

  • if it has some sort of logical structure to it can be it sorted? so that a binary search can be used to skip over a lot of pages?

  • if it's a propritary, in-memory, database engine, can the data be better indexed/clustered to not require so many memory page hits?

  • if they're image textures, can commonly used textures be located near each other?

Do you really need all 300 MiB of the memory before you can do anything? You cannot service request without all that data back in memory?

Otherwise: scheduled task at 6 ᴀᴍ to wake it up.

黎歌 2024-07-26 00:28:55

就成本而言,最便宜和最简单的解决方案可能就是为该服务器购买更多 RAM,然后您可以完全禁用页面文件。 如果您运行 32 位 Windows,只需购买 4GB RAM。 然后整个地址空间将由物理内存支持,并且页面文件无论如何都不会执行任何操作。

In terms of cost, the cheapest and easiest solution is probably just to buy more RAM for that server, and then you can disable the page file entirely. If you're running 32-bit Windows, just buy 4GB of RAM. Then the entire address space will be backed with physical memory, and the page file won't be doing anything anyway.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文