处理大字符串与内存使用有关的技巧

发布于 2024-08-17 19:08:22 字数 1095 浏览 5 评论 0原文

如何强制收缩数据表和/或列表以便有效释放内存？目前，我正在每次循环迭代时从数据集中删除已处理的行，但我不确定内存是否被释放。

for (int i = m_TotalNumberOfLocalRows - 1; i >= 0; i--)
{
   dr = dt.Rows[i];
   // Do stuff
   dt.Rows.Remove(dr);
}

如果这不会缩小数据表在内存中的占用空间，我可以使用列表。我仅使用 DataTable 作为 DataRows 的存储，我可以使用任何内存不足且能够在每 x 迭代释放内存的集合或存储机制。

谢谢。

编辑： 在阅读了什么是一些好的.NET分析器？后进行了一些内存分析之后，我我们发现内存的主要消耗者是字符串。

在此过程中我们会进行大量的用户输出，内存消耗大约在 170MB-230MB 之间循环，峰值约为 300MB。我使用初始大小为 20971520 的 StringBuilder 来保存正在发生的事情的输出/日志，在处理完记录总数的百分之一后，我将 DevExpress MemoEdit 控件的 Text 属性设置为 StringBuilder.ToString ()。我发现这种方法比将 StringBuilder.ToString() 附加到 MemoEdit.Text 更快（显然 StringBuilder 的逻辑在附加和设置 MemoEdit.Text 之间是不同的）

我还发现，而不是重新创建StringBuilder(20971520) 它更容易占用内存，并且执行速度更快，只需 StringBuilder.Remove(0, StringBuilder.Length)

您是否可以分享一些技巧来提高处理大型字符串时的性能（日志它写出的包含日志的文件大约为 12.2MB，大约有 30 000 条记录）？

注意：我已经更改了问题的标题和标签。
旧标题：如何强制收缩数据表和/或列表以释放内存？
旧标签：列出数据表c#.net内存

原文

How can I force a shrink of a DataTable and/or List so that I can free memory efficiently? I am currently removing the processed row from the DataSet every iteration of the loop, but I'm not sure if the memory is being released.

for (int i = m_TotalNumberOfLocalRows - 1; i >= 0; i--)
{
   dr = dt.Rows[i];
   // Do stuff
   dt.Rows.Remove(dr);
}

If this doesn't shrink the DataTable's footprint in memory I can use a List. I'm only using the DataTable as a store for the DataRows, I could use whatever collection or storage mechanism that would be low on memory and be able to release memory every x iterations.

Thank you.

Edit:
After doing some memory profiling after reading What Are Some Good .NET Profilers? I've discovered that the main consumer of memory are strings.

We are doing lots of user output during this process, and the memory consumption is cycling between approximately 170MB-230MB, peaking at around 300MB. I'm using a StringBuilder with an initial size of 20971520 to hold the output/log of what is happening and after one percent of the total number of records have been processed I'm setting a DevExpress MemoEdit control's Text property to the StringBuilder.ToString(). I've found that this method is quicker than appending the StringBuilder.ToString() to the MemoEdit.Text (obviously the logic w.r.t. the StringBuilder is different between appending and setting the MemoEdit.Text)

I've also found that instead of recreating the StringBuilder(20971520) it's easier on memory and quicker to execute to just StringBuilder.Remove(0, StringBuilder.Length)

Are there any tips that you could share to improve performance when working with large strings (the log file that it written out that contains the log is approximately 12.2MB for about 30 000 records)?

Note: I have changed the title of the question and the tags.
Old title:How can I Force a Shrink of a DataTable and/or a List to Release Memory?
Old tags: list datatable c# .net memory

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

谎言月老 2024-08-24 19:08:22

除非您遇到内存问题，否则不要尝试通过调用垃圾收集器手动释放它。运行时会为您处理它，并且 99% 的时间比您尝试猜测最佳时间的效率更高。

您必须记住的是，当您调用 GC.Collect() 时，它会针对垃圾收集的所有级别运行并“整理”所有需要释放的对象。您很可能会花费处理器时间等来处理当时不需要完成的事情。

如果您绝对需要，命令是 GC.Collect()

http://msdn.microsoft.com/en-us/library/xe0c2357.aspx

http://www.developer.com/net/csharp/article.php/3343191/C-Tip-Forcing-Garbage -Collection-in-NET.htm

回复收藏 0 原文

三生池水覆流年 2024-08-24 19:08:22

尝试强制执行垃圾收集器传递：

GC.Collect();

如果您想确保在代码继续执行之前所有对象都已完成，请

GC.WaitForPendingFinalizers();

在 GC.Collect() 之后立即

调用编辑： 正如下面评论中提到的那样，人们普遍认为直接调用垃圾收集器是一种不好的做法。尽管如此，这仍然可以达到释放已删除行的未使用内存的目标。

Try forcing a garbage collector pass:

GC.Collect();

If you want to make sure, that all objects are finalized before your code execution continues, call

GC.WaitForPendingFinalizers();

right after GC.Collect()

EDIT: As people mentioned in the comments below, it is widely considered a bad practice to call the Garbage Collector directly. Nevertheless, this sould achieve the goal of freeing the unnused memory of your deleted rows.

回复收藏 0 原文