当前位置：文江博客话题详情

Python 结构体的内存大小

发布于 2024-08-02 10:24:41 字数 606 浏览 4 评论 0原文

32位和64位平台上Python数据结构的内存大小有参考吗？

如果没有，那么将其放在 SO 上就好了。越详尽越好！那么以下 Python 结构使用了多少字节（取决于 len 和相关的内容类型）？

int
float
引用
str
unicode 字符串
tuple
list
dict >
set
array.array
numpy.array
deque
新式类对象
旧式类对象
...以及我忘记的一切！

（对于只保留对其他对象的引用的容器，我们显然不想计算项目本身的大小，因为它可能是共享的。）

此外，有没有一种方法可以获取对象在运行时使用的内存（递归地）或不）？

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

你げ笑在眉眼 2024-08-09 10:24:41

之前的问题的建议对此是使用 sys.getsizeof()，引用：

>>> import sys
>>> x = 2
>>> sys.getsizeof(x)
14
>>> sys.getsizeof(sys.getsizeof)
32
>>> sys.getsizeof('this')
38
>>> sys.getsizeof('this also')
48

你可以采用这种方法：

>>> import sys
>>> import decimal
>>> 
>>> d = {
...     "int": 0,
...     "float": 0.0,
...     "dict": dict(),
...     "set": set(),
...     "tuple": tuple(),
...     "list": list(),
...     "str": "a",
...     "unicode": u"a",
...     "decimal": decimal.Decimal(0),
...     "object": object(),
... }
>>> for k, v in sorted(d.iteritems()):
...     print k, sys.getsizeof(v)
...
decimal 40
dict 140
float 16
int 12
list 36
object 8
set 116
str 25
tuple 28
unicode 28

2012-09-30

python 2.7（linux，32位）：

decimal 36
dict 136
float 16
int 12
list 32
object 8
set 112
str 22
tuple 24
unicode 32

python 3.3（linux，32位）

decimal 52
dict 144
float 16
int 14
list 32
object 8
set 112
str 26
tuple 24
unicode 26

2016-08-01

OSX，Python 2.7.10（默认，2015年10月23日，19:19： 21) 达尔文上的 [GCC 4.2.1 兼容 Apple LLVM 7.0.0 (clang-700.0.59.5)]

decimal 80
dict 280
float 24
int 24
list 72
object 16
set 232
str 38
tuple 56
unicode 52

The recommendation from an earlier question on this was to use sys.getsizeof(), quoting:

>>> import sys
>>> x = 2
>>> sys.getsizeof(x)
14
>>> sys.getsizeof(sys.getsizeof)
32
>>> sys.getsizeof('this')
38
>>> sys.getsizeof('this also')
48

You could take this approach:

>>> import sys
>>> import decimal
>>> 
>>> d = {
...     "int": 0,
...     "float": 0.0,
...     "dict": dict(),
...     "set": set(),
...     "tuple": tuple(),
...     "list": list(),
...     "str": "a",
...     "unicode": u"a",
...     "decimal": decimal.Decimal(0),
...     "object": object(),
... }
>>> for k, v in sorted(d.iteritems()):
...     print k, sys.getsizeof(v)
...
decimal 40
dict 140
float 16
int 12
list 36
object 8
set 116
str 25
tuple 28
unicode 28

2012-09-30

python 2.7 (linux, 32-bit):

decimal 36
dict 136
float 16
int 12
list 32
object 8
set 112
str 22
tuple 24
unicode 32

python 3.3 (linux, 32-bit)

decimal 52
dict 144
float 16
int 14
list 32
object 8
set 112
str 26
tuple 24
unicode 26

2016-08-01

OSX, Python 2.7.10 (default, Oct 23 2015, 19:19:21) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin

decimal 80
dict 280
float 24
int 24
list 72
object 16
set 232
str 38
tuple 56
unicode 52

回复收藏 0 原文

甲如呢乙后呢 2024-08-09 10:24:41

这些答案都收集浅层尺寸信息。我怀疑这个问题的访问者最终会在这里寻求回答这个问题：“内存中这个复杂的对象有多大？”

这里有一个很好的答案：https://goshippo.com/blog/ measure-real-size-any-python-object/

妙语：

import sys

def get_size(obj, seen=None):
    """Recursively finds size of objects"""
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        size += sum([get_size(i, seen) for i in obj])
    return size

像这样使用：

In [1]: get_size(1)
Out[1]: 24

In [2]: get_size([1])
Out[2]: 104

In [3]: get_size([[1]])
Out[3]: 184

如果你想更深入地了解Python的内存模型，这里有一篇很棒的文章，其中有类似的“总大小”代码片段作为较长解释的一部分： https://code.tutsplus.com/tutorials/understand-how-much-memory-your-python-objects-use--cms-25609

These answers all collect shallow size information. I suspect that visitors to this question will end up here looking to answer the question, "How big is this complex object in memory?"

There's a great answer here: https://goshippo.com/blog/measure-real-size-any-python-object/

The punchline:

import sys

def get_size(obj, seen=None):
    """Recursively finds size of objects"""
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        size += sum([get_size(i, seen) for i in obj])
    return size

Used like so:

In [1]: get_size(1)
Out[1]: 24

In [2]: get_size([1])
Out[2]: 104

In [3]: get_size([[1]])
Out[3]: 184

If you want to know Python's memory model more deeply, there's a great article here that has a similar "total size" snippet of code as part of a longer explanation: https://code.tutsplus.com/tutorials/understand-how-much-memory-your-python-objects-use--cms-25609

回复收藏 0 原文

一个人的旅程 2024-08-09 10:24:41

我一直很高兴使用 pympler 来完成此类任务。它与许多版本的 Python 兼容——asizeof 模块尤其可以追溯到 2.2！

例如，使用 Hughdbrown 的示例，但在开头使用 from pympler import asizeof ，在结尾使用 print asizeof.asizeof(v) ，我看到（MacOSX 10.5 上的系统 Python 2.5 ）：

$ python pymp.py 
set 120
unicode 32
tuple 32
int 16
decimal 152
float 16
list 40
object 0
dict 144
str 32

显然这里有一些近似值，但我发现它对于足迹分析和调整非常有用。

I've been happily using pympler for such tasks. It's compatible with many versions of Python -- the asizeof module in particular goes back to 2.2!

For example, using hughdbrown's example but with from pympler import asizeof at the start and print asizeof.asizeof(v) at the end, I see (system Python 2.5 on MacOSX 10.5):

$ python pymp.py 
set 120
unicode 32
tuple 32
int 16
decimal 152
float 16
list 40
object 0
dict 144
str 32

Clearly there is some approximation here, but I've found it very useful for footprint analysis and tuning.

回复收藏 0 原文

潦草背影 2024-08-09 10:24:41

尝试内存分析器。
内存分析器

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

Try memory profiler.
memory profiler

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

回复收藏 0 原文

陌上芳菲 2024-08-09 10:24:41

您也可以使用 guppy 模块。

>>> from guppy import hpy; hp=hpy()
>>> hp.heap()
Partition of a set of 25853 objects. Total size = 3320992 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11731  45   929072  28    929072  28 str
     1   5832  23   469760  14   1398832  42 tuple
     2    324   1   277728   8   1676560  50 dict (no owner)
     3     70   0   216976   7   1893536  57 dict of module
     4    199   1   210856   6   2104392  63 dict of type
     5   1627   6   208256   6   2312648  70 types.CodeType
     6   1592   6   191040   6   2503688  75 function
     7    199   1   177008   5   2680696  81 type
     8    124   0   135328   4   2816024  85 dict of class
     9   1045   4    83600   3   2899624  87 __builtin__.wrapper_descriptor
<90 more rows. Type e.g. '_.more' to view.>

和：

>>> hp.iso(1, [1], "1", (1,), {1:1}, None)
Partition of a set of 6 objects. Total size = 560 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  17      280  50       280  50 dict (no owner)
     1      1  17      136  24       416  74 list
     2      1  17       64  11       480  86 tuple
     3      1  17       40   7       520  93 str
     4      1  17       24   4       544  97 int
     5      1  17       16   3       560 100 types.NoneType

Also you can use guppy module.

>>> from guppy import hpy; hp=hpy()
>>> hp.heap()
Partition of a set of 25853 objects. Total size = 3320992 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11731  45   929072  28    929072  28 str
     1   5832  23   469760  14   1398832  42 tuple
     2    324   1   277728   8   1676560  50 dict (no owner)
     3     70   0   216976   7   1893536  57 dict of module
     4    199   1   210856   6   2104392  63 dict of type
     5   1627   6   208256   6   2312648  70 types.CodeType
     6   1592   6   191040   6   2503688  75 function
     7    199   1   177008   5   2680696  81 type
     8    124   0   135328   4   2816024  85 dict of class
     9   1045   4    83600   3   2899624  87 __builtin__.wrapper_descriptor
<90 more rows. Type e.g. '_.more' to view.>

And:

>>> hp.iso(1, [1], "1", (1,), {1:1}, None)
Partition of a set of 6 objects. Total size = 560 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  17      280  50       280  50 dict (no owner)
     1      1  17      136  24       416  74 list
     2      1  17       64  11       480  86 tuple
     3      1  17       40   7       520  93 str
     4      1  17       24   4       544  97 int
     5      1  17       16   3       560 100 types.NoneType

回复收藏 0 原文