如何检查Python功能所消耗的总内存？

发布于 2025-02-11 07:40:21 字数 981 浏览 1 评论 0原文

我想检查python功能所消耗的总内存。我已经使用了tracemalloc，但是使用它，您只能检查当前大小和内存块的峰值大小。我希望能够测量所消耗的内存，例如在朱莉娅（Julia）中使用@btime宏，后者在返回表达式的值之前返回分配的内存。

我还尝试了类似的事情：

# importing libraries
import os
import psutil
 
# inner psutil function
def process_memory():
    process = psutil.Process(os.getpid())
    mem_info = process.memory_info()
    return mem_info.rss
 
# decorator function
def profile(func):
    def wrapper(*args, **kwargs):
 
        mem_before = process_memory()
        result = func(*args, **kwargs)
        mem_after = process_memory()
        print("{}:consumed memory: {:,}".format(
            func.__name__,
            mem_before, mem_after, mem_after - mem_before))
 
        return result
    return wrapper
 
# instantiation of decorator function
@profile
 
# main code for which
# memory has to be monitored
def func():
    x = [1] * (10 ** 7)
    y = [2] * (4 * 10 ** 8)
    del x
    return y
 
func()

但是，此解决方案仅在开始时返回消耗的内存，然后在每个后续调用中返回0。

原文

I want to check the total memory consumed by a function in python. I have used tracemalloc but using it you can only check current size and peak size of memory blocks. I would like to be able to measure the memory consumed like using the @btime macro in Julia, which returns the memory allocated before returning the value of the expression.

I tried also something like this:

# importing libraries
import os
import psutil
 
# inner psutil function
def process_memory():
    process = psutil.Process(os.getpid())
    mem_info = process.memory_info()
    return mem_info.rss
 
# decorator function
def profile(func):
    def wrapper(*args, **kwargs):
 
        mem_before = process_memory()
        result = func(*args, **kwargs)
        mem_after = process_memory()
        print("{}:consumed memory: {:,}".format(
            func.__name__,
            mem_before, mem_after, mem_after - mem_before))
 
        return result
    return wrapper
 
# instantiation of decorator function
@profile
 
# main code for which
# memory has to be monitored
def func():
    x = [1] * (10 ** 7)
    y = [2] * (4 * 10 ** 8)
    del x
    return y
 
func()

But this solution returns the consumed memory only at the beginning, and returns 0 on each subsequent call.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

你的往事 2025-02-18 07:40:21

看看MOMEME-PROFILER library 。

这里是从文档中获取的示例：

# example.py

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

if __name__ == '__main__':
    my_func()

然后在命令行类型中：

mprof run example.py
mprof plot

您将获得一个带有统计数据和一个漂亮图的表格：

您应该很容易地适应您感兴趣的功能。

Take a look at the memory-profiler library.

Here an example taken from the docs:

# example.py

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

if __name__ == '__main__':
    my_func()

Then in the command line type: