内存高效文件追加

发布于 2024-09-04 01:56:28 字数 1048 浏览 2 评论 0原文

我有几个文件，其内容需要合并到一个文件中。我有以下代码可以执行此操作...但在内存使用方面似乎效率很低...您会建议一种更好的方法吗？

Util.MoveFile 函数仅考虑跨卷移动文件

 private void Compose(string[] 文件)
   {
       字符串 inFile = "";
       字符串outFile =“c:\final.txt”;

       使用 (FileStream fsOut = new FileStream(outFile + ".tmp", FileMode.Create))
       {
           foreach（文件中的字符串 inFile）
           {
               if (!File.Exists(inFile))
               {
                   继续;
               }

               字节 [] 字节；
               使用 (FileStream fsIn = new FileStream(inFile, FileMode.Open))
               {
                   字节=新字节[fsIn.Length];
                   fsIn.Read(字节, 0, 字节.长度);
               }

               //使用(StreamReader sr = new StreamReader(inFile))
               //{
               // 文本 = sr.ReadToEnd();
               //}

               // 将段写入最终文件
               fsOut.Write(字节, 0, 字节. 长度);

               文件.删除(inFile);
           }
       }

       Util.MoveFile(outFile + ".tmp", outFile);

}

原文

i have several files whose content need to be merged into a single file. i have the following code that does this ... but it seems rather inefficient in terms of memory usage ... would you suggest a better way to do it ?

the Util.MoveFile function simply accounts for moving files across volumes

   private void Compose(string[] files)
   {
       string inFile = "";
       string outFile = "c:\final.txt";

       using (FileStream fsOut = new FileStream(outFile + ".tmp", FileMode.Create))
       {
           foreach (string inFile in files)
           {
               if (!File.Exists(inFile))
               {
                   continue;
               }

               byte[] bytes;
               using (FileStream fsIn = new FileStream(inFile, FileMode.Open))
               {
                   bytes = new byte[fsIn.Length];
                   fsIn.Read(bytes, 0, bytes.Length);
               }

               //using (StreamReader sr = new StreamReader(inFile))
               //{
               //    text = sr.ReadToEnd();
               //}

               // write the segment to final file
               fsOut.Write(bytes, 0, bytes.Length);

               File.Delete(inFile);
           }
       }

       Util.MoveFile(outFile + ".tmp", outFile);

}

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

公布 2024-09-11 01:56:28

有时调用 shell 函数比重新实现功能更好。正如 Alan 所说，您可以在 UNIX 系统上使用 CAT，或者在 Windows 上您可以使用内置的命令处理器

copy file1+file2+file3 concated_file

Sometimes its just better to call shell function than to reimplement functionality. As Alan says you can use CAT on unix systems or perhaps on windows you can use the built in command processor

copy file1+file2+file3 concated_file

回复收藏 0 原文

狼性发作 2024-09-11 01:56:28

您可以使用较小的固定大小缓冲区，如下所示：

byte[] bytes = new byte[8192]; // adjust this as needed
int bytesRead;
do {
    bytesRead = fsIn.Read(bytes, 0, bytes.Length);
    fsOut.Write(bytes, 0, bytesRead);
} while (bytesRead > 0);

这是非常不言自明的，除了最后一个块期间之外，所以基本上发生的情况是我将 8K 字节数组传递给 Read 方法，该方法返回实际读取的字节数。因此，在 Write 调用中，我传递的值介于 0 和 8192 之间。换句话说，在最后一个块上，即使我传递 8192 字节的字节数组，bytesRead 也可能只为 10，这种情况下只需写入前 10 个字节。

编辑

我编辑了我的答案，以稍微不同的方式做到这一点。我没有使用输入文件的位置来确定何时跳出循环，而是检查 bytesRead 是否大于零。此方法适用于任何类型的流到流复制，包括没有固定或已知长度的流。

You can use smaller fixed-size buffers like so:

byte[] bytes = new byte[8192]; // adjust this as needed
int bytesRead;
do {
    bytesRead = fsIn.Read(bytes, 0, bytes.Length);
    fsOut.Write(bytes, 0, bytesRead);
} while (bytesRead > 0);

This is pretty self explanatory except for during the last block so basically what's happening is I'm passing an 8K byte array to the Read method which returns the number of bytes it actually read. So on the Write call, I am passing that value which is somewhere between 0 and 8192. In other words, on the last block, even though I am passing a byte array of 8192 bytes, bytesRead might only be 10 in which case only the first 10 bytes need to be written.

EDIT

I edited my answer to do this in a slightly different way. Instead of using the input file's position to determine when to break out of the loop, I am checking to see if bytesRead is greater than zero. This method works with any kind of stream to stream copy including streams that don't have a fixed or known length.

回复收藏 0 原文

~没有更多了~