我应该重用 FileStream/BinaryWriter 对象吗?

发布于 2024-09-27 10:30:10 字数 3030 浏览 0 评论 0原文

更新:在查看发生此情况时的事件日志后,我收到消息:“服务器无法从系统非分页池进行分配,因为该池为空。”在整个日志中不断重复,直到重新启动。

我正在编写一个类,将调试信息写入文件,到目前为止,该类运行良好,但是我现在开始对我的应用程序进行压力测试(通过运行速度比正常快 1000 倍),这导致出现异常错误。

我看到的问题是,经过很长一段时间(4 小时以上)后,我的应用程序崩溃了,并且似乎 Windows 也随之崩溃;我无法再打开 Windows 资源管理器或任何其他应用程序。系统重新启动似乎可以解决该问题,但是当我执行此操作时,我正在写入的文件是空白的。

这让我认为问题可能与打开的文件句柄有关;也许 Windows 不知何故达到了打开文件句柄的限制?

那么,相关问题就来了;这是将数据写入文件的主要函数。如您所见,每次调用此函数时都会创建 FileStream 和 BinaryWriter 对象,并将其包装在 using 语句中以确保它们正确关闭/处置。

/// <summary>
/// This is called after changing any
/// stats data, or on initial startup.
/// It saves the current stats to file.
/// </summary>
public void UpdateStatsData()
{
    lock (this.lockObject)
    {
        using (FileStream fileStream = new FileStream(Constants.StatsFile, FileMode.Create, FileAccess.Write, FileShare.None, 128, FileOptions.WriteThrough))
        {
            using (BinaryWriter binWriter = new BinaryWriter(fileStream))
            {
                binWriter.Write(this.serverStats.APM);
                binWriter.Write(this.serverStats.AverageJackpotWin);
                binWriter.Write(this.serverStats.AverageWinnings);
                binWriter.Write(this.serverStats.NumberOfGamesPlayed);
                binWriter.Write(this.serverStats.NumberOfJackpots);
                binWriter.Write(this.serverStats.RunningPercentage);
                binWriter.Write(this.serverStats.SiteID);
                binWriter.Write(this.serverStats.TotalJackpotsValue);
                binWriter.Write(this.serverStats.TotalStaked);
                binWriter.Write(this.serverStats.TotalWinnings);
            }
        }
    }
}
  1. 是否有可能,当非常快速地调用此函数时,可能会导致文件句柄缓慢建立并最终超过 Windows 的最大值?

  2. 一种可能的解决方案包括将 FileStream 和 BinaryWriter 对象设为类的私有成员变量,在构造函数中创建它们,然后在每次调用时覆盖数据。一种

/// <summary>
/// This should be called after changing any
/// stats data, or on initial startup.
/// It saves the current stats to a serialized file.
/// </summary>
public void UpdateStatsData()
{
    lock (this.lockObject)
    {
        // Seek to the beginning of the file.
        this.binWriter.BaseStream.Seek(0, SeekOrigin.Begin);

        // Write the stats data over the existing data.
        this.binWriter.Write(this.serverStats.APM);
        this.binWriter.Write(this.serverStats.AverageJackpotWin);
        this.binWriter.Write(this.serverStats.AverageWinnings);
        this.binWriter.Write(this.serverStats.NumberOfGamesPlayed);
        this.binWriter.Write(this.serverStats.NumberOfJackpots);
        this.binWriter.Write(this.serverStats.RunningPercentage);
        this.binWriter.Write(this.serverStats.SiteID);
        this.binWriter.Write(this.serverStats.TotalJackpotsValue);
        this.binWriter.Write(this.serverStats.TotalStaked);
        this.binWriter.Write(this.serverStats.TotalWinnings);
    }
}

然而,虽然它可能更快并且仅意味着使用一个 FileStream,但如何确保 FileStream 和 BinaryWriter 在应用程序关闭时正确关闭/处置?

Update: After looking in the event log at around the time this occurred, I get the message: "The server was unable to allocate from the system nonpaged pool because the pool was empty." repeated continually throughout the log, until it was rebooted.

I am writing a class that writes debugging information to a file, up until now the class has worked fine, however I am now starting to stress-test my application (by running it at 1000x times faster than normal) and this has caused an unusual error to occur.

The problem I am seeing is that after a long period of time (4 hours+) my application crashes and seems to take out Windows with it; I can no longer open up Windows Explorer or any other application. A system reboot seems to solve the issue, however when I do the file I am writing to is blank.

This makes me think that perhaps the issue is related to open file handles; perhaps Windows is reaching it's limit of open file handles somehow?

So, here comes the related question; here is the main function that writes data to the file. As you can see, FileStream and BinaryWriter objects are created with each call to this function, wrapped in using statements to ensure they are properly Closed/Disposed.

/// <summary>
/// This is called after changing any
/// stats data, or on initial startup.
/// It saves the current stats to file.
/// </summary>
public void UpdateStatsData()
{
    lock (this.lockObject)
    {
        using (FileStream fileStream = new FileStream(Constants.StatsFile, FileMode.Create, FileAccess.Write, FileShare.None, 128, FileOptions.WriteThrough))
        {
            using (BinaryWriter binWriter = new BinaryWriter(fileStream))
            {
                binWriter.Write(this.serverStats.APM);
                binWriter.Write(this.serverStats.AverageJackpotWin);
                binWriter.Write(this.serverStats.AverageWinnings);
                binWriter.Write(this.serverStats.NumberOfGamesPlayed);
                binWriter.Write(this.serverStats.NumberOfJackpots);
                binWriter.Write(this.serverStats.RunningPercentage);
                binWriter.Write(this.serverStats.SiteID);
                binWriter.Write(this.serverStats.TotalJackpotsValue);
                binWriter.Write(this.serverStats.TotalStaked);
                binWriter.Write(this.serverStats.TotalWinnings);
            }
        }
    }
}
  1. Is it possible that this function, when called very rapidly, could cause file handles to slowly build up and eventually exceed Windows' maximum?

  2. A possible solution involves making the FileStream and BinaryWriter objects private member variables of the class, creating them in the constructor, and then overwriting the data with each call.

.

/// <summary>
/// This should be called after changing any
/// stats data, or on initial startup.
/// It saves the current stats to a serialized file.
/// </summary>
public void UpdateStatsData()
{
    lock (this.lockObject)
    {
        // Seek to the beginning of the file.
        this.binWriter.BaseStream.Seek(0, SeekOrigin.Begin);

        // Write the stats data over the existing data.
        this.binWriter.Write(this.serverStats.APM);
        this.binWriter.Write(this.serverStats.AverageJackpotWin);
        this.binWriter.Write(this.serverStats.AverageWinnings);
        this.binWriter.Write(this.serverStats.NumberOfGamesPlayed);
        this.binWriter.Write(this.serverStats.NumberOfJackpots);
        this.binWriter.Write(this.serverStats.RunningPercentage);
        this.binWriter.Write(this.serverStats.SiteID);
        this.binWriter.Write(this.serverStats.TotalJackpotsValue);
        this.binWriter.Write(this.serverStats.TotalStaked);
        this.binWriter.Write(this.serverStats.TotalWinnings);
    }
}

However, while it may be quicker and only mean using one FileStream, how do I ensure that the FileStream and BinaryWriter are Closed/Disposed properly on application shutdown?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

独﹏钓一江月 2024-10-04 10:30:10

FileStream 构造函数的参数组合对我来说看起来很可疑(假设所有线程都记录到同一个文件 (Constants.StatsFile):

  • FileMode.Create = 如果您要删除此方法中的每个条目,则始终创建该文件(可以尝试 OpenOrCreateAppend
  • FileOptions。 .WriteThrough = 无缓存 - 强制磁盘旋转并强制线程等待磁盘 - 慢

我的猜测:您调用此方法的速度比它完成的速度快得多。 >lock 语句等待上一个调用来删除文件,写入文件,并将其完全刷新到磁盘一段时间后,

假设您不打算每次都删除日志文件 。尝试这种组合,看看情况是否会变得更好,并至少摆脱 WriteThrough 因为这将使该方法更快:

using (FileStream fileStream = new FileStream(Constants.StatsFile, FileMode.Append, 
        FileAccess.Write, FileShare.None, 128, FileOptions.SequentialScan))

The combination of parameters to the FileStream constructor look suspect to me (assuming that all threads log to the same file (Constants.StatsFile):

  • FileMode.Create = Always create the file. overwrite if it exists. you are deleting all previous logs with each entry into this method (might try OpenOrCreate or Append)
  • FileOptions.WriteThrough = no caching - force the disk to spin and force the thread to wait for the disk - slow

My guess: you are calling this method much more quickly than it can complete. Each call backs up on the lock statement waiting for the previous call to delete the file, write to it, and completely flush it to disk. After awhile you just run out of memory.

Assuming you didn't intend to delete the log file each time try this combination and see if things get better and at a minimum get rid of WriteThrough as that will make this method much faster:

using (FileStream fileStream = new FileStream(Constants.StatsFile, FileMode.Append, 
        FileAccess.Write, FileShare.None, 128, FileOptions.SequentialScan))
居里长安 2024-10-04 10:30:10

在 Windows 中,耗尽非分页池内存是一个非常严重的问题。此后没有什么好事发生,驱动程序将无法完成其工作,需要重新启动才能从中恢复。

当然,用户模式程序(即托管程序)导致这种情况发生是不正常的。 Windows 通过为进程提供有限的可用系统资源配额来保护自己免受这种情况的影响。其中有很多,10,000 个句柄的限制是一个明显的限制,如果程序泄漏句柄,就会经常发生这种情况。

非分页池中的内存由驱动程序专门分配。他们需要这种宝贵的内存,因为他们在设备中断时使用内存。无法从页面文件映射内存的关键时刻。池很小,这是必要的,因为它永久占用 RAM。这取决于您机器的 RAM 量,对于具有 1 GB RAM 的机器,通常最大为 256 MB。您可以在 TaskMgr.exe 的“性能”选项卡中查看其当前大小。我现在正在给它一个不错的解决方法,它当前显示 61 MB。

显然,您的程序正在使计算机上的驱动程序消耗过多的非页面池内存。或者它正在泄漏,可能是由于您进行剧烈运动而引起的。 Windows 无法阻止这种情况,配额与进程相关,而不是与驱动程序相关。你必须找到行为不端的司机。它将是与文件系统或磁盘关联的一个。您现在可能已经猜到了,导致此类问题的一个非常常见的原因是您的病毒扫描程序。

Running out of non-paged pool memory is a very serious mishap in Windows. Nothing good happens after that, drivers will fail to do their job, a reboot is required to recover from this.

Of course, it isn't normal for a user mode program (a managed one at that) to cause this to happen. Windows protects itself against this by giving a process a limited quota of the available system resources. There are many of them, a limit of 10,000 handles is an obvious one that strikes pretty often if a program leaks handles.

Memory from the non-paged pool is exclusively allocated by drivers. They need that kind of precious memory because they use memory at device interrupt time. A critical time where it isn't possible to map memory from the paging file. The pool is small, it needs to be because it permanently occupies RAM. It depends on the amount of RAM your machine has, typically 256 MB max for a machine with 1 GB of RAM. You can see its current size in TaskMgr.exe, Performance tab. I'm giving it a decent workaround right now, it is currently showing 61 MB.

Clearly your program is making a driver on your machine consume too much non-page pool memory. Or it is leaking, possibly induced by the heavy workout you give it. Windows is powerless to prevent this, quotas are associated with processes, not drivers. You'll have to find the driver that misbehaves. It would be one that's associated with the file system or the disk. A very common one that causes trouble like this is, you probably guessed it by now, your virus scanner.

淤浪 2024-10-04 10:30:10

大部分代码对我来说看起来都很好——像您一样重新创建 FileStreams 应该没有问题。

唯一让我惊讶的是你的 lockObject 不是静态的。这可能是一个大问题——该类的多个实例将导致发生阻塞,这意味着您可能会遇到由多个线程同时运行相同代码而导致的一些奇怪情况。谁知道,在负载下您可能会同时创建数千个打开的文件句柄。

Most of this code looks fine to me -- you should have no problem re-creating the FileStreams like you are.

The only thing that jumps out at me is that your lockObject is not static. That's potentially a big problem -- multiple instances of the class will cause blocking not to occur, which means you might be running into some strange condition caused by multiple threads running the same code at the same time. Who knows, under load you could be creating thousands of open file handles all at the same time.

帅冕 2024-10-04 10:30:10

我认为第一个在手柄闭合方面没有任何问题。我做第二个;特别是您所询问的问题。您可以使您的类成为一次性的,然后理想地在“受控”关闭期间关闭它,同时根据文件对象的终结器来处理异常关闭期间的问题,但我不确定您是否修复了正确的问题问题。

打开文件句柄的哪些测量结果证实了您对这就是问题的怀疑?当您确实打开大量文件时,怀疑打开的文件句柄是合理的,但“修复”这一点是愚蠢的,除非 A)检查代码表明它显然会存在此问题(此处不是这种情况)或 B)您已经表明这样的文件句柄确实太高了。

应用程序崩溃时是否会在事件查看器中留下异常?

I see nothing wrong with the first in terms of handle closure. I do with the second; specifically the very issues you ask about. You could make your class disposable and then ideally close it during a "controlled" shut-down, while depending on the file object's finaliser to take care of matters during exceptional shut-down, but I'm not sure you're fixing the right issue.

What measurements of open file handles confirm your suspicion that this is the issue? It's reasonable to suspect open file handles when you are indeed opening lots of files, but it's foolish to "fix" that unless either A) examining the code shows it will obviously have this problem (not the case here) or B) you've shown that such file handles are indeed too high.

Does the app leave an exception in the event viewer on crashing?

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文