如何强制我的 std::map 释放使用的内存?

发布于 2024-08-28 11:55:44 字数 558 浏览 3 评论 0原文

我正在使用 std::map,而且我似乎无法将内存释放回操作系统。看起来,

int main(){
  aMap m;

  while(keepGoing){
    while(fillUpMap){
       //populate m
    }
    doWhatIwantWithMap(m);
    m.clear();//doesnt free memory back to OS

    //flush some buffered values into map for next iteration
    flushIntoMap(m);
  }
}

每个 (fillUpmap) 分配大约 1gig,所以我非常有兴趣在它耗尽我所有的内存之前将其返回到我的系统。

我对 std::vector 也经历过同样的情况,但我可以通过与空的 std::vector 进行交换来强制它释放。这不适用于地图。

当我使用 valgrind 时,它说所有内存都已释放,因此这不是泄漏问题,因为运行后所有内容都很好地清除了。

编辑:

同花必须出现在清除之后。

I'm using a std::map, and I can't seem to free the memory back to the OS. It looks like,

int main(){
  aMap m;

  while(keepGoing){
    while(fillUpMap){
       //populate m
    }
    doWhatIwantWithMap(m);
    m.clear();//doesnt free memory back to OS

    //flush some buffered values into map for next iteration
    flushIntoMap(m);
  }
}

Each (fillUpmap) allocates around 1gig, so I'm very much interested in getting this back to my system before it eats up all my memory.

Ive experienced the same with std::vector, but there I could force it to free by doing a swap with an empty std::vector. This doesn't work with map.

When I use valgrind it says that all memory is freed, so its not a problem with a leak, since everything is cleared up nicely after a run.

edit:

The flush has to appear after the clear.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(7

凉栀 2024-09-04 11:55:44

m.clear() 将内存释放回堆,但堆实现通常不会将内存释放回操作系统(即使可以,但碎片等问题也会使其变得困难)。

这就是默认分配器的工作方式,如果您为映射指定了自己的分配器,它可能有自己的缓存。然而,即使在这种情况下,它也应该被缓存以便立即重用。

地图不像矢量那样有容量与大小的概念。

m.clear() releases memory back to the heap, but it's common for heap implementations to not release that back to the OS (even when they can, issues such as fragmentation make it hard).

This is how the default allocator works, if you've specified your own allocator for the map, it might have its own cache. However, even in that case, it should be cached in order to be reused immediately anyway.

Maps don't have a concept of capacity vs size the way vectors do.

丿*梦醉红颜 2024-09-04 11:55:44

这种行为是正常的,运行时库会将映射类分配的内存保留给进程使用,这样下次需要分配内存时就不必再访问操作系统了。这是一个运行时库优化。

That behaviour is normal, the runtime library keeps that memory allocated by the map class avaiable to the process so that the next time it needs to allocate memory it does not have to go to the operating system. It is an run time library optimisation.

水溶 2024-09-04 11:55:44

如果您在堆上创建映射(通过 new),删除它将释放它使用的所有内存。

If you create the map on the heap (via new), deleting it will free up any memory it used.

错々过的事 2024-09-04 11:55:44

也许你可以创建一个自定义分配器,或者只使用 Boost 的 池库。

Maybe you could create a custom allocator, or just use Boost's pool library.

第七度阳光i 2024-09-04 11:55:44

当我将一些数据放入 std::map 然后调用 std::map::clear() 时,我做了一个简单的测试。

typedef std::map<int, unit_t,std::less<int>,  
     my_allocator<std::pair<const int, unit_t>, 4 > > contaner_t;
contaner_t keys;
keys[1] = 10;
keys[2] = 20;
keys[3] = 30;
keys.clear();

这是我的测试中插入 printf 的结果:

Allocator # 4, Memory consumption:      56     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (allocated   :       56)
Allocator # 4, Memory consumption:     168     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (deallocated :       56),
Allocator # 4, Memory consumption:      56     (deallocated :       56),
Allocator # 4, Memory consumption:       0     (deallocated :       56),

我认为您可能还应该检查您的分配器行为,但我认为您的默认 std::allocator 实际上会按照您的预期释放内存,但该内存不会返回到操作系统。顺便问一下你使用什么操作系统?

这里的问题是你如何测量似乎无法将内存释放回操作系统。以及你如何确定我可以通过与空的交换来强制它释放std::vector。 您真的确定分配的内存实际上会返回到操作系统吗?

I have done a simple test when I put some data in a std::map and then call std::map::clear().

typedef std::map<int, unit_t,std::less<int>,  
     my_allocator<std::pair<const int, unit_t>, 4 > > contaner_t;
contaner_t keys;
keys[1] = 10;
keys[2] = 20;
keys[3] = 30;
keys.clear();

this is the result of printf inserted in my test:

Allocator # 4, Memory consumption:      56     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (allocated   :       56)
Allocator # 4, Memory consumption:     168     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (deallocated :       56),
Allocator # 4, Memory consumption:      56     (deallocated :       56),
Allocator # 4, Memory consumption:       0     (deallocated :       56),

I think you probably should also check your allocator behavior but I think your default std::allocator in fact deallocates memory as you expect but that memory is not returned to the OS. By the way what OS do you use?

The question here is how you measure can't seem to free the memory back to the OS. and how can you be sure that I could force it to free by doing a swap with an empty std::vector. Are you really sure that allocated memory in fact goes back to OS?

执笔绘流年 2024-09-04 11:55:44

我猜你用的是Linux。

如果你确实需要最小化应用程序的内存占用,你可以调用 清除地图后的 malloc_trim() 。我还建议您看一下 mallopt() 手册页- 它有一些关于为什么你的代码可以保留内存而不是将其返回到操作系统的提示。

I guess you are on Linux.

If you really need to minimize the memory footprint of your application, you can call malloc_trim() after clearing the map. I would also recommend taking a glance at mallopt() manpage - it has some hints as to why your code could keep memory instead of returning it to the OS.

﹏雨一样淡蓝的深情 2024-09-04 11:55:44

Linux 编程接口的 7.1.2 节清楚地解释了内存释放:

In general, free() doesn’t lower the program break, but instead adds the block of
memory to a list of free blocks that are recycled by future calls to malloc(). This is
done for several reasons:
- The block of memory being freed is typically somewhere in the middle of the
heap, rather than at the end, so that lowering the program break is not possible.
- It minimizes the number of sbrk() calls that the program must perform. (As
noted in Section 3.1, system calls have a small but significant overhead.)
- In many cases, lowering the break would not help programs that allocate large
amounts of memory, since they typically tend to hold on to allocated memory
or repeatedly release and reallocate memory, rather than release it all and then
continue to run for an extended period of time.

Section 7.1.2 of The Linux Programming Interface explains clearly about memory deallocation:

In general, free() doesn’t lower the program break, but instead adds the block of
memory to a list of free blocks that are recycled by future calls to malloc(). This is
done for several reasons:
- The block of memory being freed is typically somewhere in the middle of the
heap, rather than at the end, so that lowering the program break is not possible.
- It minimizes the number of sbrk() calls that the program must perform. (As
noted in Section 3.1, system calls have a small but significant overhead.)
- In many cases, lowering the break would not help programs that allocate large
amounts of memory, since they typically tend to hold on to allocated memory
or repeatedly release and reallocate memory, rather than release it all and then
continue to run for an extended period of time.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文