加载和释放NSImage时内存持续增加
我遇到的问题是,我的应用程序在连续加载图像文件时会大量消耗内存,达到“崩溃点”。例如,考虑以下代码,该代码重复加载和释放 15MB JPEG 文件(用于测试目的的大文件大小):
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
for(int i=0; i<1000; i++) {
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
}
由于有足够的可用系统内存,它在前几个文件中执行得很快,但最终系统崩溃了在我所说的“打击点”。在这里,我相信系统释放了足够的内存来加载下一个图像,因此性能最终会变慢。此外,现在其他应用程序运行缓慢,因为系统必须释放这些占用但现在未使用的内存。
对我来说有意义的是,如果它分配内存,然后让系统释放它,以便活动监视器中的“真实内存”统计数据保持较小,而不是通过连续的加载/释放迭代进入千兆字节。实际上,我可能永远不会在这一点上结束,但奇怪的是,当任何时间所需的实际驻留内存通常都很小时,我的活动监视器“Real Mem”统计数据最终超过了所有其他应用程序。我最初认为这是某种内存泄漏或缓存问题,但它似乎与应用程序中的积极内存分配和系统上的惰性内存释放更相关(并不是说如果操作系统有该策略就没有任何问题 - 如果是这样)事实上它的工作方式)。也许我完全错过了一些东西。
有没有更好的方法来重复加载图像,而不会出现这种占用且不主动释放内存使用行为的情况?也许有一种方法可以强制减少应用程序内存占用,或者有一种方法可以更智能地了解如何将图像加载到相同的对象或内存位置?我的目标是加载图像,处理它(获取缩略图,更改图像格式等),在内存中删除它,然后再次执行 - 所有这些都不会观察到内存增长。
--
后续:
巴各斯,谢谢。包装 NSAutoreleasePool 确实解决了加载同一文件时迭代内存增长的问题:
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
for(int i=0; i<1000; i++) {
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
[apool drain];
}
但是,它并没有解决图像释放后内存保持增加的问题(并且 NSAutoreleasePool 被耗尽)。例如,当加载 15MB JPEG 图像时,“Real Mem”内存从 8MB 稳定状态跳至约 25MB,然后保持在那里。 (我的应用程序只有一个 Interface Builder 按钮,该按钮具有连接到仅调用我复制的 for 循环的方法的 IBAction )。我希望在 for 循环完成后(或者如果只加载和释放一张图像),“Real Mem”统计数据将减少回名义应用程序级别的内存使用量。
调用 NSImage 功能时也可以加载后台的其他内容,这似乎是合理的,这可以增加内存。然而,不同大小的图像(15MB、30MB、50MB 等)会按比例增加应用程序中的内存,这让我相信它不仅仅是这样的分配。
此外,如果我尝试连续加载单独的图像(例如,15MBjpeg-1.jpg、15MBjpeg-2.jpg 等),内存有时会为加载的每个新图像进行复合。例如,如果连续加载两个图像,则加载/释放后应用程序的“Real Mem”内存使用量现在约为 50MB,并且根据我的观察,它永远不会减少。加载后续图像时,此行为会继续存在,因此应用程序在加载多个大图像后可能会占用数百 MB 的“Real Mem”内存。
有趣的是,如果我一遍又一遍地重新加载相同的图像,稳态内存不会增加。这是否表明正在进行某种缓存?同样,我的目标是在不增加内存的情况下对几个不同的图像文件进行批处理。提前致谢。
哦,我正在研究 Heapshot Analysis 文章 ATM,但至少想发布我的进度并看看是否有其他输入。
--
后续#2
bbum,感谢这篇精彩的文章。我用我的测试程序运行了 Instruments Allocations,但没有发现任何堆增长。正如所引用的博客文章中所述,我的方法是,1)单击 Interface Builder 界面上的“加载和释放图像”按钮(调用加载/释放行为),2)每隔几秒单击几次“标记堆”在“工具分配”中,然后 3) 重复 1) 和 2)。
使用这种方法,随着时间的推移,Heapshots 在堆增长列中始终报告 0 字节(每 5 秒点击 3 次,持续 15 秒),这意味着没有从基线 Heapshot 分配额外的内存。另外,在“统计”窗格中,每次单击“加载并释放映像”按钮时都会有 13.25MB 的 Malloc,但 Live Bytes 为 0 字节,这意味着它已完全释放。如果我单击“加载和释放映像”按钮三次,则映像的总字节数报告为 39.75MB (3 * 13.25MB),这意味着已分配 39.75MB,但由于实时字节数为 0,因此已完全释放。分配图快速上升并立即下降,因为这是一个相当快的操作。内存的稳态使用没有泄漏也没有增长,这一切似乎都是有道理的。
但是,现在我该怎么办,我的“Real Mem”统计数据仍然很高?我知道活动监视器不是调试内存问题的标准。但是,“Real Mem”仍然保持高位,直到我关闭程序,然后“Real Mem”全部回到“免费”类别,这对我来说很奇怪。
我通过在相同的方法中复制代码,使用两个图像(15MBjpeg-1.jpg、15MBjpeg-2.jpg)测试了相同的方法,并且我再次观察到没有堆增长。显然,已经进行和释放了更多的分配。然而,现在“Real Mem”的增量大约是仅加载和释放一张图像的情况的两倍。同样,在测试程序的稳定状态下它不会减少。
我还能做些什么吗?以下是单个图像加载/释放的测试代码,供任何想要尝试的人使用(只需将 IB 按钮连接到 openFiles):
#import "TestMemory.h" // only declares -(IBAction) openFiles:(id)sender;
@implementation TestMemory
-(IBAction) openFiles:(id)sender {
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
[apool drain];
}
@end
感谢您的阅读。 :)
--
FOLLOW-UP #3
bbum,不,我没有启用垃圾收集(GC 在项目设置中设置为“不支持”)。我使用 vmmap 和仪器分配中的 VM Tracker 栏研究了内存。当您提到 VM Instruments 时,我假设您指的是 VM Tracker 数据,因为它报告的信息与 vmmap 相同。使用“加载和释放映像”按钮打开单个映像后,一些重要的内存数字包括以下内容(来自 VM Tracker):
Type % ofRes ResSize VirtSize Res% %AllDirty DirtySize
__TEXT 38% 33.84MB 80.45MB 42% 0%&n公共服务提供商; 0Bytes
*脏*
32% 28.23MB 114.99MB 24% ;100% 17.11MBMALLOC_LARGE 14% 13.25MB 13.25MB 100% 0% nbsp; ; 4KB
碳 11% 9.86MB 9.86MB 100% 2 0 % 3.46MB
VM_ALLOCATE 9% 8.43MB 48.17MB 18% 49% 8.43MB 48.17MB 18% 49% ; 8.43MB
...
有趣的是,单个映像的后续加载/释放将脏类型和 VM_ALLOCATE 类型的“驻留大小”增加了~.3MB,并且这些类型的“脏大小”也随着时间的推移而增加。 (VM_ALLOCATE 似乎是 Dirty 的子集)。其他类型似乎不会随着后续加载/发布而改变。
我不确定要从这些数据中删除什么,或者如何使用它来使程序释放内存。看起来 VM_ALLOCATE 类型可能是未释放的块,但这只是猜测。底层 NSImage init 实现的某些部分是否可能保存图像文件的缓存并且不会释放它?再次,如前所述,令我感兴趣的是,与第一次加载/释放相比,同一图像文件的每次后续加载/释放几乎不消耗资源(CPU、磁盘研磨等)和挂钟时间。提前致谢。
I have a problem where my application aggressively consumes memory to a "thrashing point" with successive image file loads. For example, consider the following code, which repeatedly loads and releases a 15MB JPEG file (large file size for test purposes):
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
for(int i=0; i<1000; i++) {
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
}
It performs quickly for the first several since there is plenty of free system memory, but eventually the system comes to its knees at what I'm calling the "thrashing point." Here, I believe the system relieves just enough memory to load the next image, so performance ends up being slow. Additionally, now other applications run slowly because the system has to free up this hogged, but now unused, memory.
What would make sense to me is if it would allocate the memory and then have the system free it so that my "Real Mem" statistic in Activity Monitor stays small rather than heading into the gigabytes with successive load/release iterations. In practice, I may never end up at this point, but it seems odd that my Activity Monitor "Real Mem" statistic eventually exceeds all other applications when the actual resident memory required at any time is generally small. I originally thought this was some sort of memory leak or caching issue, but it seems more related to aggressive memory allocation in the application and lazy memory freeing on the system (not that there's anything wrong if the OS has that policy--if that is in fact the way it works). Perhaps I'm missing something altogether though.
Is there a better way to repeatedly load images without this hogging-and-not-proactively-relieving memory usage behavior? Perhaps there is a way to force the application memory footprint down, or there is a way to be smarter about how images are loaded into the same objects or memory locations? My goal is to load an image, process it (get a thumbnail, change the image format, etc), get rid of it in memory, and then do it again--all without this observed memory growth.
--
FOLLOW-UP:
Bavarious, thanks. A wrapping NSAutoreleasePool does resolve the iterative memory growth when loading the same file:
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
for(int i=0; i<1000; i++) {
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
[apool drain];
}
However, it does not solve the issue where the memory stays increased after the image is released (and the NSAutoreleasePool drained). For example, when loading my 15MB JPEG image, the "Real Mem" memory jumps from the 8MB steady state to about 25MB and then stays there. (My application only has an Interface Builder button that has a wired IBAction to a method that invokes only the for loop I copied). I would expect that after the for loop finishes (or if only one image is loaded and released) that the "Real Mem" statistic would decrease back to the nominal application level memory usage.
It seems reasonable that other things in the background can be loaded as well when invoking NSImage functionality, which can increase the memory. However, different sized images (15MB, 30MB, 50MB, etc) increase memory proportionally in the application, which leads me to believe it's more than such allocation.
Further, if I try to load separate images in succession (say, 15MBjpeg-1.jpg, 15MBjpeg-2.jpg, etc.) the memory sometimes compounds for each new image loaded. For example, if two images are loaded in succession, the "Real Mem" memory usage for the application after the loading/releasing is now roughly 50MB, and it never decreases--based on my observation. This behavior continues when loading subsequent images so that the application can creep into hundreds of MB of "Real Mem" memory usage after loading several large images.
Interestingly, if I reload the same image over and over again, the steady state memory does not increase. Does this indicate some sort of caching that's going on? Again, my goal is to batch process through several different image files without this growth in memory. Thanks in advance.
Oh, and I'm looking into the Heapshot Analysis article ATM, but wanted at least to post my progress and see if there's other input.
--
FOLLOW-UP #2
bbum, thanks for the great article. I ran Instruments Allocations with my test program and did not find any Heap Growth. As on the referenced blog post, my approach was to, 1) click my "Load and Release Image" button on my Interface Builder interface (which invokes the load/release behavior), 2) click "Mark Heap" several times every few seconds in Instruments Allocations, and then 3) repeat 1) and 2).
Using this approach, the Heapshots consistently reported 0 Bytes in the Heap Growth column over time (3 clicks every 5 sec for 15 seconds), meaning that there is no additional memory allocated from the baseline Heapshot. Additionally, in the Statistics pane, there is a Malloc of 13.25MB each time I click the "Load and Release Image" button, but Live Bytes is 0 Bytes, meaning that it has been fully released. If I click the "Load and Release Image" button three times, the Overall Bytes for the image reports 39.75MB (3 * 13.25MB), meaning that 39.75MB was allocated, but fully released since the Live Bytes is 0. The Allocations graph spikes up quickly and comes right back down, since it's a pretty quick operation. It all seems to make sense that there is no leak and no growth in the steady state use of memory.
But, now what do I do that my "Real Mem" statistic is still high? I know that Activity Monitor is not the standard for debugging memory issues. But, "Real Mem" still stays high until I close the program, and then the "Real Mem" all goes back into the "Free" category, which is bizarre to me.
I tested this same approach with two images (15MBjpeg-1.jpg, 15MBjpeg-2.jpg) by just duplicating the code in the same method, and I observe no Heap Growth again. Obviously more allocations were been made and released. However, now the "Real Mem" sits at roughly twice the increase as in the case of loading and releasing only one image. And again, it doesn't decrease in the test program's steady state.
Is there anything further I can do? Here is the test code for a single image load/release for anyone who wants to give it a try (just wire an IB button to openFiles):
#import "TestMemory.h" // only declares -(IBAction) openFiles:(id)sender;
@implementation TestMemory
-(IBAction) openFiles:(id)sender {
NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
[image release];
[apool drain];
}
@end
Thanks for reading. :)
--
FOLLOW-UP #3
bbum, no I don't have garbage collection enabled (GC is set to Unsupported in the project settings). I investigated the memory using vmmap and the VM Tracker bar in Instruments Allocations. I assumed you meant the VM Tracker data when you said VM Instruments, since it reports the same information as vmmap. After opening a single image using my "Load and Release Image" button, some significant memory numbers include the following (from VM Tracker):
Type %ofRes ResSize VirtSize Res% %AllDirty DirtySize
__TEXT 38% 33.84MB 80.45MB 42% 0% 0Bytes
*Dirty*
32% 28.23MB 114.99MB 24% 100% 17.11MBMALLOC_LARGE 14% 13.25MB 13.25MB 100% 0% 4KB
Carbon 11% 9.86MB 9.86MB 100% 20% 3.46MB
VM_ALLOCATE 9% 8.43MB 48.17MB 18% 49% 8.43MB
...
Interestingly, subsequent load/releases of a single image increase the "Resident Size" of the Dirty and VM_ALLOCATE types by ~.3MB, and the "Dirty Size" of these types also increases over time. (VM_ALLOCATE seems to be a subset of Dirty). No other types appear to change with subsequent load/releases.
I am unsure exactly what to take away from this data, or how I can use it to make the program release the memory. It seems like the VM_ALLOCATE type may be the chunk that is not being freed, but that's only speculation. Is it possible that the some portion of the underlying NSImage init implementation saves a cache of the image file and won't release it? Again, as stated earlier, it intrigues me that each subsequent load/release of the same image file hardly consumes resources (CPU, disk grinding, etc) and wall clock time compared to the first load/release. Thanks in advance.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
巴瓦里厄斯所说的话;您是否尝试过用 NSAutoreleasePool 包围它?
这是一个经典的微基准。虽然它肯定表明存在问题,但问题实际上可能是基准与预期的现实世界模式相差如此之大,以至于错误在于基准。
在高效设计的应用程序中,它不会多次从磁盘读取相同的图像数据。
这是 Heapshot 分析。
(感谢您的后续工作;很有帮助!)
您描述的症状听起来像是虚拟机泄漏(某些东西在不进行分配的情况下消耗地址;例如映射内存)或未修剪的缓存(包含虚拟机分配) )。
您是否启用了 GC?如果是这样,很可能是因为 GC 阈值未触发。收集器不知道将非 GC 中的真正大量分配记入 GC 区域。如果您强制收集,它将解决这个特定的边缘情况。
尝试查看 VM 工具或在命令行中使用 vm_map 来查看应用程序中消耗地址空间的内容。
What Bavarious said; have you tried surrounding it with an
NSAutoreleasePool
.That is a classic micro-benchmark. While it certainly indicates a problem, the problem may actually be that the benchmark is divergent from expected real world patterns so much that the bug lies with the benchmark.
In an efficiently designed application, it would not read the same image data from disk more than once.
This is a prime candidate for Heapshot analysis.
(Thanks for the follow-ups; helpful!)
The symptoms you describe sound like either a VM leak (something is consuming addresses without doing allocations; mapped memory, for example) or a cache that isn't being pruned (that contains VM allocations).
do you have GC enabled? If so, this could easily be because the GC threshold isn't triggered. The collector doesn't know to credit Really Large Allocations from non-GC to the GC zone. If you force collections, it'll work around this particular edge case.
Try having a look at the VM instrument or use vm_map at the command line to have a look at what is consuming the address space within your app.