如何找出使用 PIL 函数导致内存错误的原因?

发布于 2024-12-09 12:29:18 字数 2346 浏览 0 评论 0原文

我正在使用 PIL(python 图像库)进行一些图像处理,特别是我将图像拼接在一起。

我的代码似乎对于一些少量图像工作得很好,但有时我会得到一个MemoryError

对我来说特别奇怪的部分是我没有对位图像进行操作,它全部适用于 10kb 以下的 <10 个图像。

我对 Image.resize 进行了大量调用,但令我惊讶的是,其中存在重大问题。

这是堆栈轨迹:

Traceback (most recent call last):                                                                                                                                               
  File "test.py", line 15, in <module>                                                                                                                                           
    pprint(scale_matrix_down((90,90), [inpt]))                                                                                                                                   
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 105, in scale_matrix_down                                                                                            
    return [shrinkRow(row, row_width_overflow(row)) for row in matrix]                                                                                                           
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 103, in shrinkRow                                                                                                    
    rest         = [shrinkIm(im, pixels_per_im) for im in row[remaining_pixels:]]                                                                                                
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 110, in shrinkIm                                                                                                     
    return im.resize((im.size[0] - num_pix, im.size[1] - num_pix))                                                                                                               
  File "/Library/Python/2.7/site-packages/PIL/Image.py", line 1302, in resize                                                                                                    
    im = self.im.resize(size, resample)                                                                                                                                          
MemoryError

请记住,图像均小于 90x90 像素。

我陷入了死胡同,真的不知道如何继续。我可以做什么来释放内存?我应该调用 del 运算符还是可以做一些更简单的事情? 预先感谢您的帮助!

I am using PIL (python image library) to do some image manipulation, specifically I am stitching images together.

My code seems to work fine for some images in small quantities, but sometime I and getting a MemoryError.

The part that is particularly strange to me is that I am not doing manipulations on bit images, its all work with <10 images under 10kb.

I am making a lot of calls to Image.resize, but I am surprised that there are significant issues from that.

Here is the stack track:

Traceback (most recent call last):                                                                                                                                               
  File "test.py", line 15, in <module>                                                                                                                                           
    pprint(scale_matrix_down((90,90), [inpt]))                                                                                                                                   
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 105, in scale_matrix_down                                                                                            
    return [shrinkRow(row, row_width_overflow(row)) for row in matrix]                                                                                                           
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 103, in shrinkRow                                                                                                    
    rest         = [shrinkIm(im, pixels_per_im) for im in row[remaining_pixels:]]                                                                                                
  File "/Users/jeremykarmel/Desktop/Python/merger.py", line 110, in shrinkIm                                                                                                     
    return im.resize((im.size[0] - num_pix, im.size[1] - num_pix))                                                                                                               
  File "/Library/Python/2.7/site-packages/PIL/Image.py", line 1302, in resize                                                                                                    
    im = self.im.resize(size, resample)                                                                                                                                          
MemoryError

Keep in mind the images are all less than 90x90 pixels.

I am very much at a dead end and really not sure how to proceed. What can I do to release the memory? Should I be calling the del operator or is there something simpler I can do?
Thanks in advance for your help!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

初见你 2024-12-16 12:29:18

解释实际发生的情况:

  1. 您尝试将图像大小调整为负宽度
  2. PIL 尝试分配负大小的图像以适合调整大小的图像
  3. PIL 调用 malloc 使用负数的内存来保存负大小的图像
  4. malloc 需要一个 size_t ,它是未签名。因此,负内存请求被解释为一个非常大的数字,而不是负数。
  5. malloc 无法分配大的东西,因此它会产生一个错误
  6. 对于 PIL,这看起来与内存不足错误完全相同,因此它会这样报告

因此无效的输入不会导致泄漏或其他不良行为。它们会导致错误的 malloc 请求并放弃。 PIL 可以检查负大小,从而生成更好的错误消息。也许他们认为这不值得,因为如果你这样做,它已经产生了一条错误消息。

事实上,实际上耗尽内存是很困难的,因为操作系统会非常努力地保持进程继续运行,例如使用虚拟内存。您的系统在内存耗尽之前就会陷入困境。所以我发现大多数内存不足错误都是由负内存量请求引起的。据我记得,我遇到的唯一真正的内存不足错误是在 Java 中,可能是由于它使用了虚拟机。

To explain what actually happens:

  1. You try to resize an image to a negative width
  2. PIL tries to allocate a negatively sized image to fit the resized image
  3. PIL calls malloc with a negative amount of memory to hold to negatively sized image
  4. malloc takes a size_t which is unsigned. As a result, the negative memory request gets interpreted as a very large number rather then a negative number.
  5. malloc cannot allocate something large, so it produces an error
  6. To PIL this looks exactly like an out of memory error so it reports it as such

So the invalid inputs aren't causing leaks or other bad behavior. They result in an bad malloc request and give up. PIL could check for negative sizes, and thus produce a better error message. Perhaps they figure its not worth it because it already produces an error message if you do that.

As it happens, actually running out memory is hard because the OS will try pretty hard to keep a process going such as by using virtual memory. Your system will bog down before it gets to a point of running out memory. So I've found that most out-of-memory errors are caused by requests for negative amounts of memory. As far as I recall, the only real out-of-memory errors I've gotten have been in Java probably due to its use of a virtual machine.

难忘№最初的完美 2024-12-16 12:29:18

事实证明这实际上并不是内存错误。正如 Winston Ewert 指出的那样,我实际上是将负参数输入到图像调整大小方法中。

尽管 python 文档说内存错误是针对内存问题的,但当您给出负参数来调整大小时,会抛出此错误。我的怀疑是,由于 PIL 大量利用 C 库,这些可能会导致无效输入的泄漏,并且由于该库不进行边界检查,因此错误只会冒出来。

It turns out this is not actually a memory error. As Winston Ewert pointed out I was in fact feeding negative parameters into an images resize method.

Even though the python documentation says that memory errors are for problems with memory, this error gets thrown when you give negative params to resize. My suspicion is that because PIL heavily leverages C libraries, those can lead to leaks with invalid input and since the library does not do bounds checking, that the error simply bubbles up.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文