释放 Solaris/Linux 中分配的内存

发布于 2024-09-03 15:47:15 字数 509 浏览 7 评论 0原文

我编写了一个小程序,并在 Solaris/Linux 平台下编译它,以测量将此代码应用于我的应用程序的性能。

程序是这样编写的,最初使用sbrk(0)系统调用,我获取了堆区域的基地址。之后,我使用 malloc 系统调用分配了 1.5 GB 内存,然后使用 memcpy 系统调用将 1.5 GB 内容复制到分配的内存区域。然后,我释放了分配的内存。

释放后,我再次使用sbrk(0)系统调用来查看堆大小。

这是我有点困惑的地方。在 Solaris 中,即使我释放了分配的内存(接近 1.5 GB),进程的堆大小仍然很大。但我在Linux中运行相同的应用程序,释放后,我发现进程的堆大小等于分配前的堆内存大小1.5 GB。

我知道 Solaris 不会立即释放内存,但我不知道如何调整 Solaris 内核以在 free() 系统调用后立即释放内存。

为什么我在Linux下没有遇到同样的问题?

I have written a small program and compiled it under Solaris/Linux platform to measure the performance of applying this code to my application.

The program is written in such a way, initially using a sbrk(0) system call, I have taken base address of the heap region. After that I have allocated 1.5 GB of memory using a malloc system call, Then I used a memcpy system call to copy 1.5 GB of content to the allocated memory area. Then, I freed the allocated memory.

After freeing, I used the sbrk(0) system call again to view the heap size.

This is where I get a little confused. In Solaris, even though I freed the memory allocated (nearly 1.5 GB), the heap size of the process is huge. But I run the same application in Linux, after freeing, I found that the heap size of the process is equal to the size of the heap memory before allocation of 1.5 GB.

I know Solaris does not free memory immediately, but I don't know how to tune the Solaris kernel to immediately free the memory after the free() system call.

Why don't I have the same problem under Linux?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

路弥 2024-09-10 15:47:15

我得到了我提出的问题的答案。

应用程序内存分配器:

C 和 C++ 开发人员必须手动管理内存分配和释放内存。默认内存分配器位于 libc 库中。


请注意,执行 free() 后,释放的空间可供应用程序进一步分配,而不返回给系统。仅当应用程序终止时,内存才会返回给系统。这就是应用程序的进程大小通常永远不会减小的原因。但对于长时间运行的应用程序,应用程序进程大小通常保持稳定状态,因为释放的内存可以重用。如果情况并非如此,则应用程序很可能正在泄漏内存,也就是说,分配的内存已被使用,但不再使用时从未释放,并且应用程序不会跟踪指向分配的内存的指针 - 基本上丢失了。

当频繁发生并发 malloc 或 free 操作时,libc 中的默认内存分配器不适合多线程应用程序,特别是对于多线程 C++ 应用程序。这是因为创建和销毁 C++ 对象是 C++ 应用程序开发风格的一部分。当使用默认的 libc 分配器时,堆由单个堆锁保护,导致默认分配器由于 malloc 或 free 操作期间的严重锁争用而无法针对多线程应用程序进行扩展。使用 Solaris 工具很容易检测到此问题,如下所示。

首先,使用 prstat -mL -p 查看应用程序是否在锁上花费了很多时间;看看LCK栏目。例如:

-bash-3.2# prstat -mL -p 14052
   PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
 14052 root     0.6 0.7 0.0 0.0 0.0  35 0.0  64 245  13 841   0 test_vector_/721
 14052 root     1.0 0.0 0.0 0.0 0.0  35 0.0  64 287   5 731   0 test_vector_/941
 14052 root     1.0 0.0 0.0 0.0 0.0  35 0.0  64 298   3 680   0 test_vector_/181
 14052 root     1.0 0.1 0.0 0.0 0.0  35 0.0  64 298   3  1K   0 test_vector_/549
 ....

它表明应用程序花费了大约 35% 的时间来等待锁。

然后,使用 plockstat(1M) 工具查找应用程序正在等待的锁。例如,跟踪进程 ID 14052 的应用程序 5 秒,然后使用 c++filt 实用程序过滤输出以分解 C++ 符号名称。 (c++filt 实用程序随 Sun Studio 软件提供。)如果应用程序不是 C++ 应用程序,则不需要通过 c++filt 进行过滤。

-bash-3.2#  plockstat -e 5 -p 14052 | c++filt
Mutex block
Count     nsec   Lock                         Caller
-------------------------------------------------------------------------------
 9678 166540561 libc.so.1‘libc_malloc_lock   libCrun.so.1‘void operator 
 delete(void*)+0x26

 5530 197179848 libc.so.1‘libc_malloc_lock   libCrun.so.1‘void*operator 
 new(unsigned)+0x38

......

从前面的内容可以看出,堆锁 libc_malloc_lock 的竞争非常激烈,并且很可能是导致扩展问题的原因。 libc 分配器的扩展问题的解决方案是使用改进的内存分配器,例如 libumem 库。

另请访问:http://developers.sun.com/solaris/articles/solaris_memory.html

感谢所有尝试回答我的问题的人,
桑托什。

I got the answer for the question that i have asked.

Application Memory Allocators:

C and C++ developers must manually manage memory allocation and free memory. The default memory allocator is in the libc library.

Libc
Note that after free()is executed, the freed space is made available for further allocation by the application and not returned to the system. Memory is returned to the system only when the application terminates. That's why the application's process size usually never decreases. But for a long-running application, the application process size usually remains in a stable state because the freed memory can be reused. If this is not the case, then most likely the application is leaking memory, that is, allocated memory is used but never freed when no longer in use and the pointer to the allocated memory is not tracked by the application—basically lost.

The default memory allocator in libc is not good for multi-threaded applications when a concurrent malloc or free operation occurs frequently, especially for multi-threaded C++ applications. This is because creating and destroying C++ objects is part of C++ application development style. When the default libc allocator is used, the heap is protected by a single heap-lock, causing the default allocator not to be scalable for multi-threaded applications due to heavy lock contentions during malloc or free operations. It's easy to detect this problem with Solaris tools, as follows.

First, use prstat -mL -p to see if the application spends much time on locks; look at the LCK column. For example:

-bash-3.2# prstat -mL -p 14052
   PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
 14052 root     0.6 0.7 0.0 0.0 0.0  35 0.0  64 245  13 841   0 test_vector_/721
 14052 root     1.0 0.0 0.0 0.0 0.0  35 0.0  64 287   5 731   0 test_vector_/941
 14052 root     1.0 0.0 0.0 0.0 0.0  35 0.0  64 298   3 680   0 test_vector_/181
 14052 root     1.0 0.1 0.0 0.0 0.0  35 0.0  64 298   3  1K   0 test_vector_/549
 ....

It shows that the application spend about 35 percent of its time waiting for locks.

Then, using the plockstat(1M) tool, find what locks the application is waiting for. For example, trace the application for 5 seconds with process ID 14052, and then filter the output with the c++filt utility for demangling C++ symbol names. (The c++filt utility is provided with the Sun Studio software.) Filtering through c++filt is not needed if the application is not a C++ application.

-bash-3.2#  plockstat -e 5 -p 14052 | c++filt
Mutex block
Count     nsec   Lock                         Caller
-------------------------------------------------------------------------------
 9678 166540561 libc.so.1‘libc_malloc_lock   libCrun.so.1‘void operator 
 delete(void*)+0x26

 5530 197179848 libc.so.1‘libc_malloc_lock   libCrun.so.1‘void*operator 
 new(unsigned)+0x38

......

From the preceding, you can see that the heap-lock libc_malloc_lock is heavily contended for and is a likely cause for the scaling issue. The solution for this scaling problem of the libc allocator is to use an improved memory allocator like the libumem library.

Also visit: http://developers.sun.com/solaris/articles/solaris_memory.html

Thanks for all who tried to answer my question,
Santhosh.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文