key 删除后 ruby​​ hash 内存泄漏

发布于 2024-11-06 18:26:33 字数 716 浏览 1 评论 0原文

你好, 我无法成功如何在哈希中删除键后释放内存。当我从哈希中删除密钥时,手动调用 GC.start 后也不会释放内存。这是预期的行为还是当从哈希中删除键并且这些对象在某处泄漏时 GC 不会释放内存?如何在 Ruby 中删除哈希中的键并在内存中取消分配它?

示例:

irb(main):001:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 4748
irb(main):002:0> a = {}
=> {}
irb(main):003:0> 1000000.times{|i| a[i] = "test #{i}"}
=> 1000000
irb(main):004:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 140340
irb(main):005:0> 1000000.times{|i| a.delete(i)}
=> 1000000
irb(main):006:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 140364
irb(main):007:0> GC.start
=> nil
irb(main):008:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 127076

PS:我使用 ruby​​ 1.8.7。我也尝试过 ruby​​ 1.9.2,但并没有更好。

Helo,
i can't succeed how to release memory after key deletion in hash. When I delete key from Hash, memory is not released nor after calling GC.start manually. Is this expected behavior or GC does not release memory when keys are deleted from Hash and these objects are leaking somewhere? How can I delete key in Hash in Ruby and unallocate it also in memory?

Example:

irb(main):001:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 4748
irb(main):002:0> a = {}
=> {}
irb(main):003:0> 1000000.times{|i| a[i] = "test #{i}"}
=> 1000000
irb(main):004:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 140340
irb(main):005:0> 1000000.times{|i| a.delete(i)}
=> 1000000
irb(main):006:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 140364
irb(main):007:0> GC.start
=> nil
irb(main):008:0> `ps -o rss= -p #{Process.pid}`.to_i
=> 127076

PS: I use ruby 1.8.7. I've tried also ruby 1.9.2, but it was not better.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

没企图 2024-11-13 18:26:33

请参阅 Stackoverflow:Malloc 和 Free 的工作原理

,原因有很多(在上面的引文中详细说明)实际上没有内存管理器将内存释放回操作系统。

为了让您看到进程的变化,Ruby 解释器的 C 部分中的底层 mallocfree 需要将内存归还给主机操作系统。这不会发生,但在 Ruby 级别,对象已被 GC 处理并位于解释器中本地保存的空闲列表中。

See Stackoverflow: How Malloc and Free Work

For a variety of good reasons (spelled out in the citation above) virtually no memory manager releases memory back to the operating system.

For you to see the process change, the underlying malloc and free in the C part of the Ruby interpreter would need to give the host OS back the memory. That's not going to happen, but at the Ruby level, the objects have been GC'ed and are in a locally kept free list in the interpreter.

眉黛浅 2024-11-13 18:26:33

作为一名长期使用多种语言进行此类工作的高级开发人员,我的想法如下:

虽然我认为在使用 C 等编译语言时您的意图是好的,但细粒度的开发人员控制的内存管理并不适合深入了解 Ruby、Python 和 Perl 等语言的工作原理。

Perl、Ruby 和 Python 等脚本语言使我们摆脱了内存管理的烦恼。这就是我们喜欢它们的原因之一。如果我们有可用的内存,他们就会使用所需的内存来完成工作。内存管理控制的丧失是开发速度和调试方便性的权衡;我们没有,也不需要担心。如果我需要它,我会使用 C 或汇编语言。

至于假设这是内存泄漏,我认为这有点天真或自以为是。像你提到的那样的内存泄漏将是一个严重的泄漏,并且,对于尽可能多的基于 Ruby 的应用程序和网站,有人很久以前就会注意到它。因此,当我看到一些没有意义的东西时,作为对自己的健全性检查,我总是认为我首先在代码中做错了什么,然后我会看看我对某些东西如何工作的假设,如果这些看起来仍然不错,我会去寻找其他有类似问题的人,看看他们是否有解决方案。而且,如果问题是该语言的核心问题,我将深入研究源代码或与一些核心开发人员交谈,并询问我是否对我所看到的感到疯狂。我以前发现过低级错误,但它们一直是极端情况,在提到任何内容之前我花了几天时间进行挖掘,因为我不想像我的同事一样提交错误报告立即与 Apple 联系,然后发现这是他的代码中的错误。

我关于在释放时将内存返回给系统的总体想法是,它会产生额外的开销,这些开销可能会在浪费 CPU 周期的下一个操作中逆转,这是解释型语言和脚本语言无法承受的,因为它们不如编译型语言快。开始于。我认为对于该语言来说,假设它需要重复分配一大块内存(如果它必须这样做一次),这是公平的事情,特别是对于像 Ruby 这样的面向对象语言。在这一点上,保留以前使用过的内存就很有意义。

而且,从总体上看,考虑到我们的盒子里通常有多少空闲内存,分配 1,000,000 个该大小的数组元素并不是很多内存。我更关心在内存中维护数组中 1,000,000 个元素的需要,并建议同行认真考虑使用数据库。您可能有充分的商业理由将其全部保存在 RAM 中。如果是这样,请将主机上的 RAM 最大化,应该没问题。

As a senior developer who's been doing this stuff a long time in a lot of languages, here's my thoughts:

While I think your intention is good when using a compiled language such as C, fine-grained developer-controlled memory management doesn't fit into how languages like Ruby, Python and Perl do things.

Scripting languages like Perl, Ruby and Python insulate us from the worries of memory management. That's one of the reasons we like them. If we have the memory available, they'll use what they need in order to get the job done. The loss of memory management control is a tradeoff for speed of development and ease of debugging; We don't have it, and don't need to worry about it. If I need it I'll use C or assembly language.

As far as assuming it's a memory leak, well, I think that's a bit naive or presumptuous. A memory leak like you mention would be a significant leak, and, with as many Ruby-based apps and sites as there are, someone would have noticed it a long time ago. So, as a sanity check for myself when I see something that doesn't make sense, I always figure I'm doing something wrong in my code first, then I'll take a look at my assumptions about how something works, and if those still seem sound, I'll go looking for other people who have similar problems and see if they have solutions. And, if the problem is something that would be core to the language, I'll dig into the source or talk to some of the core developers and ask if I'm nuts with what I'm seeing. I've found low-level bugs before but they've been corner cases, and I spent a couple days digging around before I mentioned anything, because I didn't want to be like a peer of mine who'd file a bug report with Apple immediately, then find out it was a bug in his code.

My overall thinking regarding returning memory back to the system on a deallocation, is it incurs additional overhead that might be reversed in the next operation wasting CPU cycles, which interpreted and scripting languages can not afford since they're not as fast as compiled languages to begin with. I think it's a fair thing for the language to assume that it will need to repeatedly allocate a big block of memory if it's had to do it once, especially with an OO language like Ruby. At that point it makes a lot of sense to hold on to the memory previously used.

And, in the big scheme of things, allocating 1,000,000 array elements of that size isn't a lot of memory considering how much we routinely have free in our boxes. I would be more concerned about the need to maintain 1,000,000 elements in an array in memory and would recommend to a peer that they should look seriously at using a database. You might have a sound business reason for holding it all in RAM. If so, max out the RAM on the host and you should be fine.

悲凉≈ 2024-11-13 18:26:33

这些对象应该被垃圾收集。如果您要再次创建它们,该进程不应显着增长,因为它有所有空白空间。然而,Ruby 不会将该内存释放回操作系统,因为它假设将来可能再次需要那么多内存。

这是一个相当简单的解释,但基本上,您所看到的都是正常的。

The objects should be garbage collected. If you were to create them again, the process should not grow significantly, because it has all that empty space. However, Ruby does not release that memory back to the OS, because it assumes that it is likely to need that much memory again in the future.

This is a rather simplistic explanation, but basically, what you are seeing is normal.

七分※倦醒 2024-11-13 18:26:33

如果您运行分配两次,或多或少可以看到@digitalross 的情况。如果确实存在这样的内存泄漏,您可能会期望内存大小加倍,但这种情况不会发生。

[~]$ irb                                         rvm:ruby-1.9.3-p0@global 
1.9.3p0 :001 > `ps -o rss= -p #{Process.pid}`.to_i
 => 8148 
1.9.3p0 :002 > a = {}
 => {} 
1.9.3p0 :003 > 1000000.times{|i| a[i] = "test #{i}"}
 => 1000000 
1.9.3p0 :004 > `ps -o rss= -p #{Process.pid}`.to_i
 => 101188 
1.9.3p0 :005 > 1000000.times{|i| a.delete(i)}
 => 1000000 
1.9.3p0 :006 > `ps -o rss= -p #{Process.pid}`.to_i
 => 90960 
1.9.3p0 :007 > GC.start
 => nil 
1.9.3p0 :008 > `ps -o rss= -p #{Process.pid}`.to_i
 => 93388 
1.9.3p0 :009 > `ps -o rss= -p #{Process.pid}`.to_i
 => 93388 
1.9.3p0 :010 > 1000000.times{|i| a[i] = "test #{i}"}
 => 1000000 
1.9.3p0 :011 > `ps -o rss= -p #{Process.pid}`.to_i
 => 140088 
1.9.3p0 :012 > 1000000.times{|i| a.delete(i)}
 => 1000000 
1.9.3p0 :013 > `ps -o rss= -p #{Process.pid}`.to_i
 => 130880 
1.9.3p0 :014 > GC.start
 => nil 
1.9.3p0 :015 > `ps -o rss= -p #{Process.pid}`.to_i
 => 104256 

第一次运行结束时,该进程报告内存大小为 93388,第二次运行后报告内存大小为 104256,内存使用量仅增加了约 10%。

What @digitalross can more or less be seen if you run the allocations twice. If there was really such a memory leak, you would expect the memory size to double, however that does not happen.

[~]$ irb                                         rvm:ruby-1.9.3-p0@global 
1.9.3p0 :001 > `ps -o rss= -p #{Process.pid}`.to_i
 => 8148 
1.9.3p0 :002 > a = {}
 => {} 
1.9.3p0 :003 > 1000000.times{|i| a[i] = "test #{i}"}
 => 1000000 
1.9.3p0 :004 > `ps -o rss= -p #{Process.pid}`.to_i
 => 101188 
1.9.3p0 :005 > 1000000.times{|i| a.delete(i)}
 => 1000000 
1.9.3p0 :006 > `ps -o rss= -p #{Process.pid}`.to_i
 => 90960 
1.9.3p0 :007 > GC.start
 => nil 
1.9.3p0 :008 > `ps -o rss= -p #{Process.pid}`.to_i
 => 93388 
1.9.3p0 :009 > `ps -o rss= -p #{Process.pid}`.to_i
 => 93388 
1.9.3p0 :010 > 1000000.times{|i| a[i] = "test #{i}"}
 => 1000000 
1.9.3p0 :011 > `ps -o rss= -p #{Process.pid}`.to_i
 => 140088 
1.9.3p0 :012 > 1000000.times{|i| a.delete(i)}
 => 1000000 
1.9.3p0 :013 > `ps -o rss= -p #{Process.pid}`.to_i
 => 130880 
1.9.3p0 :014 > GC.start
 => nil 
1.9.3p0 :015 > `ps -o rss= -p #{Process.pid}`.to_i
 => 104256 

At the end of the first run, the process reports a memory size of 93388, after the second run it reports 104256, only about a 10% increase in memory use.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文