你的程序需要多少内存? (FastMM 与 Borland MM)

发布于 2024-10-08 18:46:18 字数 759 浏览 0 评论 0原文

我最近在我的程序中看到了一个奇怪的行为。创建大量对象(500MB RAM)然后释放它们后,程序的内存占用量不会恢复到其原始大小。它仍然显示 160MB 的占用空间(私有工作集)。

正常行为?

Borland 的内存管理器不会像这样运行,所以如果可能的话请确认(或确认)这是 FastMM 的正常行为:如果您有一个方便的程序,在其中创建了一个相当复杂的 MDI子(包含多个控件/对象),您是否可以在内存中循环创建该 MDI 子级的 250 个实例(同时),然后释放它们全部并检查内存占用量。请确保这些 MDI 子项至少消耗 200-300MB 或 RAM。

特别是那些仍在使用 Delphi 7 的用户可以通过暂时禁用 FastMM 来看到差异。

谢谢


如果有人感兴趣,特别是如果你想要一些证明这不是内存泄漏(我希望这不是我的代码中的内存泄漏 - 这也是这篇文章的要点之一:检查这是否是我的错) ,以下是原始讨论:

我的程序永远不会释放内存。为什么?
如何说服内存管理器释放未使用的内存

I have seen recently a strange behavior in my program. After creating large amounts of objects (500MB of RAM) then releasing them, the program's memory footprint does not return to its original size. It still shows a footprint of 160MB (Private working set).

Normal behavior?

Borland's memory manager does not behave like this, so if possible please confirm (or infirm) this is a normal behavior for FastMM: If you have a handy program in which you create a rather complex MDI child (containing several controls/objects), can you create in a loop 250 instances of that MDI child in memory (at the same time) then release them all and check the memory footprint. Please make sure that you consume at least 200-300MB or RAM with those MDI childs.

Especially those that still using Delphi 7 can see the difference by temporary disabling FastMM.

Thanks


If anybody is interested, especially if you want some proof this is not a memory leak (I hope it is not a mem leak in my code - this is also one of the points of this post: to check if it is my fault), here are the original discussions:

My program never releases the memory back. Why?
How to convince the memory manager to release unused memory

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

羅雙樹 2024-10-15 18:46:18

亲爱的阿尔塔,我很惊讶你的猜测是多么的离谱,而且你是多么不听人们以前多次告诉过你的事情。

让我们把一些事情弄清楚。内存管理101。请仔细阅读。

当您在 Delphi 中分配内存时,会涉及到两个内存管理器。

系统内存管理器

第一个是系统内存管理器。它内置于 Windows 中,提供 4kb 大小的页面内存。

但它并不总是为您提供 RAM(或物理内存)中的内存。您的数据可以保存在硬盘上,并在每次需要访问时读回。这太慢了。

换句话说,假设您有 512Mb 的物理内存。您运行两个程序,每个程序都请求 1Gb 内存。操作系统有什么作用?

它同意这两个请求。两个应用程序均获得 1Gb 内存。两人都认为所有的记忆都“在记忆中”。但实际上,RAM 中只能保留 512Mb。其余部分存储在页面文件中,尽管您的应用程序不知道这一点。它只是工作缓慢。

工作集大小

现在,您正在测量的“工作集大小”是多少?

它是保存在 RAM 中的已分配内存的一部分。

如果您有一个分配 1Gb 内存的应用程序,而您只有 512 Mb RAM,那么它的工作集大小将为 512Mb。虽然它“使用”了1Gb内存!

当您运行另一个需要内存的应用程序时,操作系统将通过将很少使用的“内存”块移动到硬盘驱动器来自动释放一些 RAM。

您的虚拟内存分配将保持不变,但更多页面将位于硬盘驱动器上,而更少页面将位于 RAM 中。工作集大小将会减小。

由此看来,您应该已经明白,尝试最小化工作集大小是没有意义的。你一事无成。您在任何意义上都没有释放内存。您只是将数据卸载到硬盘驱动器。

但系统会在需要时自动执行此操作。在需要之前,在 RAM 中腾出空间是没有意义的。您只是减慢了应用程序的速度,仅此而已。

TLDR:“工作集大小”不是“应用程序使用了多少内存”。这是“现在准备了多少”。不要试图最小化它,你只会让事情变得更糟。

Delphi 内存管理器

操作系统为您提供 4Kb 页的虚拟内存。但通常您需要更小的块。例如,整数 4 个字节,某些结构 32 个字节。解决方案是什么?

应用程序内存管理器,例如 FastMM 或 BorlandMM 或其他。

它的工作是从操作系统的页面中分配内存,然后在需要时为您提供这些页面的小块。

换句话说,当您请求 14 字节内存时,会发生以下情况:

  1. 您向 FastMM 请求 14 字节内存。
  2. FastMM 向操作系统请求 1 页内存(4096 字节)。
  3. 操作系统授予一页内存,并用 RAM 对其进行备份(它存储在实际 RAM 中)。
  4. FastMM 保存该页面,剪切其中的 14 个字节并提供给您。

当您请求另外 14 个字节时,FastMM 只会从同一页中剪切另外 14 个字节。

当你释放内存时会发生什么?反过来也是一样:

  1. 释放 14 个字节给 FastMM。什么也没发生。
  2. 您又释放了 14 个字节。 FastMM 发现它分配的 4096 字节页面现在完全未使用。
  3. 因此它释放该页面,将其返回给系统。

值得注意的是,FastMM 不能只向系统释放 14 个字节。它必须以页为单位释放内存。在整个页面空闲之前,FastMM 无法执行任何操作。没有人可以。

那么,为什么我的工作集大小这么大,即使我发布了所有内容?

首先,您的工作集大小不是您应该测量的大小。虚拟内存消耗是。但如果你的工作集大小很大,你的虚拟内存消耗也会很高。

有什么问题吗?到这里你应该能明白了。

假设您分配了 1kb,然后分配了 3kb 内存。你分配了多少虚拟内存? 4kb,1 页。

现在您释放 3Kb。您现在使用了多少虚拟内存? 1Kb?不,它仍然是 1 页。您从系统分配的页数不能少于 1 页。您仍在使用 4096 字节的虚拟内存。

想象一下,如果你这样做 1000 次。 1kb、3kb、1kb、3kb、1kb、3kb 等等。像这样分配 1000 * 4kb = 4 mb,然后释放所有 3kb 部分。您现在使用了多少虚拟内存?

还是 4MB。因为你一开始分配了1000页。每个页面占用 1kb 和 3kb 块。即使您释放 3kb 块,1kb 块仍将继续保留您在内存中分配的每个页面。每个页面占用 4kb 的虚拟内存。

内存管理器无法神奇地将所有 1kb 块“移动”在一起。这是不可能的,因为可以从代码中的某处引用它们的虚拟地址。这不是FastMM的特点。

但为什么使用 BorlandMM 一切都会更好呢?

巧合。也许 BorlandMM 为您提供内存的方式与 FastMM 略有不同。接下来您知道,您在应用程序中更改了某些内容,BorlandMM 的行为就像 FastMM 一样。内存管理器不可能完全阻止这种称为内存碎片的影响。

那我该怎么办?

简短的回答是,没什么,直到这让你烦恼为止。

您会发现,使用现代操作系统,您并没有真正占用任何人的 RAM。根据上面的描述,当其他应用程序需要 RAM 时,操作系统会自动交换您的页面。这不应该是一个问题。

而且“过多”的记忆并没有丢失。尽管分配了页面,但每个页面有 3kb 被标记为“空闲”。下次您的应用程序需要内存时,内存管理器将使用该空间。

但如果你真的想帮助它,你应该重新组织你的分配,以便你计划保留的那些先完成,然后你即将释放的全部分配。

像这样:1kb,1kb,1kb,...,3kb,3kb,3kb...

如果您现在释放所有3kb块,您的虚拟内存消耗将显着下降。

这并不总是可能的。如果不可能,那就什么也不做。或多或少都可以。

PS:

您一开始就不应该分配 500 个表格。这显然不是一条路。解决这个问题,你甚至不需要考虑内存分配和释放。

我希望这能澄清问题,因为坦率地说,关于同一主题的四篇文章有点太多了。

Dear Altar, I'm dazzled at how off the point you are in your guesses and how you don't listen to what people told you many times before.

Let's set some things straight. Memory management 101. Please read thoroughly.

When you allocate memory in Delphi, there are two memory managers involved.

System memory manager

First one is a system memory manager. This one is built into Windows and it gives memory in 4kb sized pages.

But it doesn't always give you memory in RAM (or physical memory). Your data can be kept on the hard drive, and read back every time you need to access it. This is awfully slow.

In other words, imagine you have 512Mb of physical memory. You run two programs, each requesting 1Gb of memory. What does OS do?

It grants both requests. Both apps get 1Gb of memory each. Both think all the memory is "in memory". But in fact, only 512Mb can be kept in RAM. The rest is stored in page file, although your app does not know that. It just works slow.

Working set size

Now, what is a "working set size" you are measuring?

It's the part of the allocated memory that is kept in RAM.

If you have an application which allocates 1Gb of memory, and you only have 512 Mb of RAM, then it's working set size will be 512Mb. Although it "uses" 1Gb of memory!

When you run another application which needs memory, OS will automatically free some RAM by moving rarely used blocks of "memory" to the hard drive.

Your virtual memory allocation will stay the same, but more pages will be on the hard drive and less in RAM. Working set size will decrease.

From this, you should have understood by this point, that it's pointless to try and minimize the working set size. You're achieving nothing. You're not freeing memory in any sense. You're just offloading the data to the hard drive.

But the system will do that automatically when it needs to. And there's no point making room in RAM until it's needed. You're just slowing down your application, that's all.

TLDR: "Working set size" is not "how much memory application uses". It's "how much is ready right now". Don't try to minimize it, you're just making things worse.

Delphi memory manager

OS gives you virtual memory in pages of 4Kb. But often you need it in much smaller chunks. For instance, 4 bytes for your integer, or 32 bytes for some structure. The solution?

Application memory manager, such as FastMM or BorlandMM or others.

It's job is to allocate memory in pages from the operating system, then give you small chunks of those pages when you need it.

In other words, when you ask for 14 bytes of memory, this is what happens:

  1. You ask FastMM for 14 bytes of memory.
  2. FastMM asks OS for 1 page of memory (4096 bytes).
  3. OS grants one page of memory, backing it up with RAM (it's stored in actual RAM).
  4. FastMM saves that page, cuts 14 bytes of it and gives to you.

When you ask for another 14 bytes, FastMM just cuts another 14 bytes from the same page.

What happens when you release memory? The same thing backwards:

  1. You release 14 bytes to FastMM. Nothing happens.
  2. You release another 14 bytes. FastMM sees that the 4096 byte page it allocated is now completely unused.
  3. Therefore it releases the page, returning it to the system.

It's worth noting that FastMM cannot release just 14 bytes to the system. It has to release memory in pages. Until the whole page is free, FastMM cannot do a thing. Nobody can.

So, why is my working set size so big, even though I released everything?

First, your working set size is not what you should be measuring. Virtual memory consumption is. But if you have big working set size, your virtual memory consumption will be high too.

What's the problem? You should be able to figure out by this point.

Let's say you allocate 1kb, then 3kb of memory. How much virtual memory have you allocated? 4kb, 1 page.

Now you release 3Kb. How much virtual memory do you use now? 1Kb? No, it's still 1 page. You cannot allocate less than 1 page from the system. You're still using 4096 bytes of virtual memory.

Imagine if you do that 1000 times. 1kb, 3kb, 1kb, 3kb, 1kb, 3kb and so on. You allocate 1000 * 4kb = 4 mb like that, and then you release all the 3kb parts. How much virtual memory do you use now?

Still 4 mb. Because you allocated 1000 pages at first. Of every page you took 1kb and 3kb chunks. Even if you release 3kb chunks, 1kb chunks will continue to keep every single page you allocated in memory. And every page takes 4kb of virtual memory.

Memory manager cannot magically "move" all of your 1kb chunks together. This is impossible, because their virtual addresses can be referenced from somewhere in code. It's not a trait of FastMM.

But why with BorlandMM everything works better?

Coincidence. Maybe it just so happens that BorlandMM gives you memory in a slightly different way than FastMM does. Next thing you know, you change something in your app and BorlandMM acts just like FastMM did. It's impossible for a memory manager to completely prevent this effect, called memory fragmentation.

So what do I do?

Short answer is, not much until this bothers you.

You see, with modern operating systems, you're not really eating anyone's RAM. Per above, OS will automatically swap your pages out when it needs RAM for other applications. This should not be a concern.

And the "excessive" memory isn't lost. Although pages are allocated, 3kb of each is marked as "free". Next time your app needs memory, memory manager will use that space.

But if you really want to help it, you should reorganize your allocations so that the ones you're planning on keeping are done first, and the ones you will soon release are all allocated after that.

Like this: 1kb, 1kb, 1kb, ..., 3kb, 3kb, 3kb...

If you now release all the 3kb chunks, your virtual memory consumption will drop significantly.

This is not always possible. If it's impossible, then just do nothing. It's more or less alright like it is.

And P.S.

You shouldn't be allocating 500 forms in the first place. This is clearly not a way to go. Fix this, and you won't even have a need to think about memory allocation and releasing.

I hope this clears things up, because four posts on the same topic, frankly, is a bit too much.

洒一地阳光 2024-10-15 18:46:18

IIRC,Delphi 内存管理器不会立即将释放的内存返回给操作系统。

内存被分配为小、中、大大小的块,称为块。
这些块的内容被处理后会保留一段时间,以便以后请求另一次分配时可以使用它们。

这限制了连续分配多个对象所需的系统调用量,并有助于避免堆碎片。

IIRC, the Delphi memory manager does not immediately return free'd memory to the OS.

Memory is allocated in chunks of small, medium and large sizes, called blocks.
These blocks are kept for a while after their contents have been disposed to have them readyly available when another allocation is requested afterwards.

This limits the amount of system calls required for succesive allocation of multiple objects, and helps avoiding heap fragmentation.

蓝梦月影 2024-10-15 18:46:18

确认:Delphi 2007,默认内存管理器(应该是 FastMM 变体)。对重物的多次测试:

  1. 初始内存2Mb,峰值内存30Mb,最终内存4Mb。
  2. 初始内存 2Mb,峰值内存 1Gb,最终内存 5.5Mb。

Infirming: Delphi 2007, default memory manager (should be FastMM variation). Several tests on heavy objects:

  1. Initial memory 2Mb, peak memory 30Mb, final memory 4Mb.
  2. Initial memory 2Mb, peak memory 1Gb, final memory 5.5Mb.
妥活 2024-10-15 18:46:18

关于仍分配 160MB 的堆管理器统计信息 (GetHeapStatus) 是多少?

What are the heapmanager stats (GetHeapStatus) on the point that 160MB is still allocated?

沉默的熊 2024-10-15 18:46:18

已解决

为了确认此行为是由 FastMM 生成的(按照 Barry Kelly 的建议),我创建了第二个分配大量 RAM 的程序。一旦Windows耗尽RAM,我的程序内存利用率就恢复到原来的值。

问题解决了。特别感谢巴里·凯利,他是唯一指出真正“问题”的人。

SOLVED

To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I created a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.

Problem solved. Special thanks to Barry Kelly, the only person that pointed to the real "problem".

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文