一次给出例外,另一次则没有

发布于 2024-12-12 01:52:11 字数 551 浏览 0 评论 0原文

我使用多线程将文件复制到另一个地方。我从流中需要字节数组并处理流。在这个例子中,我使用 7 个线程来复制 3GB 文件。第一个线程可以获取字节数组,但在第二个线程发生异常“System.OutOfMemoryException” '

public void Begin()
{
    FileStream stream = new FileStream(pathToFile, FileMode.Open);
    stream.Position = (threNmb - 1) * 536870912;
    BinaryReader reader = new BinaryReader(stream);
    for (long i = 0; i < (length); i++)
    {
        source.Add(reader.ReadByte());//gives exception at i=134217728
    }
    reader.Dispose();
    reader.Close();
    stream.Dispose();
    stream.Close();
}

I use multi threading to copy file to another place.I from stream needed byte array and dispose stream.at this example i use 7 threads to copy 3gb file.1st thread can get byte array,but at 2nd thread occurs exception 'System.OutOfMemoryException'

public void Begin()
{
    FileStream stream = new FileStream(pathToFile, FileMode.Open);
    stream.Position = (threNmb - 1) * 536870912;
    BinaryReader reader = new BinaryReader(stream);
    for (long i = 0; i < (length); i++)
    {
        source.Add(reader.ReadByte());//gives exception at i=134217728
    }
    reader.Dispose();
    reader.Close();
    stream.Dispose();
    stream.Close();
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

已下线请稍等 2024-12-19 01:52:11

看起来您正在使用 List。这将是一种非常低效的数据复制方式,并且您可能会通过使用多个线程来降低其效率。此外,如果您使用多个线程中的单个列表,则您的代码已经损坏 - List 不是线程安全的。即使是这样,您也会混合来自不同线程的数据,因此您将无法重新组合原始数据。哦,如果不使用 using 语句,如果抛出异常,您将使文件句柄保持打开状态。换句话说,我建议你彻底放弃目前的做法。

相反,将文件从一个位置复制到另一个位置(假设您不能使用 File.Copy 出于某种原因),基本上应该是这样的情况:

  • 打开流进行读取
  • 打开流进行写入
  • 分配缓冲区(例如 32K)
  • 重复:
    • 从“输入”流读入缓冲区,注意调用实际读取了多少数据
    • 将刚刚读取的数据写入输出流
    • 循环直到“Read”指示输入流结束
  • 关闭两者流(带有 using 语句)

不需要将所有内容都存储在内存中。请注意,在 .NET 4 中,使用 可以使这变得更加容易Stream.CopyTo 方法。它为您执行第三步和第四步。

It looks like you're using a List<byte>. That's going to be a very inefficient way of copying data, and you're probably making it less efficient by using multiple threads. Additionally, if you're using a single list from multiple threads, your code is already broken - List<T> isn't thread-safe. Even if it were, you'd be mixing the data from the different threads, so you wouldn't be able to reassemble the original data. Oh, and by not using using statements, if an exception is thrown you're leaving file handles open. In other words, I'm advising you to completely abandon your current approach.

Instead, copying a file from one place to another (assuming you can't use File.Copy for some reason), should basically be a case of:

  • Open stream to read
  • Open stream to write
  • Allocate buffer (e.g. 32K)
  • Repeatedly:
    • Read from the "input" stream into the buffer, noting how much data the call actually read
    • Write the data you've just read into the output stream
    • Loop until your "Read" indicates the end of the input stream
  • Close both streams (with using statements)

There's no need to have everything in memory. Note that in .NET 4 this is made even easier with the Stream.CopyTo method. which does the third and fourth steps for you.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文