当对象实例化父级时，请调试内存积累：我应该如何进行？

Question

我有一个我在循环中实例化的对象，这会导致内存积累。

for l in range(10):
    
    grido = RasterModelGrid((3000, 3000), 10)
    
    # Tracking memory
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('traceback')
    
    # pick the biggest memory block
    stat = top_stats[0]
    print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
    for line in stat.traceback.format():
        print(line)
    
    del grido
    
    # Tracking memory
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('traceback')
    
    # pick the biggest memory block
    stat = top_stats[0]
    print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
    for line in stat.traceback.format():
        print(line)

由于调试输出是：

    2 memory blocks: 281156.3 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 13
    grido = RasterModelGrid((3000, 3000), 10)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/grid/raster.py", line 238
    DualUniformRectilinearGraph.__init__(
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/dual_structured_quad.py", line 187
    UniformRectilinearGraph.__init__(self, shape, spacing=spacing, origin=origin)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 654
    StructuredQuadGraphExtras.__init__(self, node_y_and_x, sort=sort)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 518
    links=StructuredQuadLayoutCython.nodes_at_link(self.shape),
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 88
    nodes_at_link = np.empty((n_links, 2), dtype=int)
2 memory blocks: 281156.3 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 13
    grido = RasterModelGrid((3000, 3000), 10)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/grid/raster.py", line 238
    DualUniformRectilinearGraph.__init__(
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/dual_structured_quad.py", line 187
    UniformRectilinearGraph.__init__(self, shape, spacing=spacing, origin=origin)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 654
    StructuredQuadGraphExtras.__init__(self, node_y_and_x, sort=sort)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 518
    links=StructuredQuadLayoutCython.nodes_at_link(self.shape),
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 88
    nodes_at_link = np.empty((n_links, 2), dtype=int)
4 memory blocks: 562312.7 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 13
    grido = RasterModelGrid((3000, 3000), 10)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/grid/raster.py", line 238
    DualUniformRectilinearGraph.__init__(
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/dual_structured_quad.py", line 187
    UniformRectilinearGraph.__init__(self, shape, spacing=spacing, origin=origin)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 654
    StructuredQuadGraphExtras.__init__(self, node_y_and_x, sort=sort)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 518
    links=StructuredQuadLayoutCython.nodes_at_link(self.shape),
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 88
    nodes_at_link = np.empty((n_links, 2), dtype=int)
4 memory blocks: 562312.7 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 13
    grido = RasterModelGrid((3000, 3000), 10)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/grid/raster.py", line 238
    DualUniformRectilinearGraph.__init__(
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/dual_structured_quad.py", line 187
    UniformRectilinearGraph.__init__(self, shape, spacing=spacing, origin=origin)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 654
    StructuredQuadGraphExtras.__init__(self, node_y_and_x, sort=sort)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 518
    links=StructuredQuadLayoutCython.nodes_at_link(self.shape),
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 88
    nodes_at_link = np.empty((n_links, 2), dtype=int)

但是，当我直接与父类实例化时：

    grido = DualUniformRectilinearGraph((3000, 3000), 10)

我没有以下：

    grido = RasterModelGrid((3000, 3000), 10)

我没有此内存问题：

2 memory blocks: 281156.3 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 16
    grido = DualUniformRectilinearGraph((3000, 3000), 10)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/dual_structured_quad.py", line 187
    UniformRectilinearGraph.__init__(self, shape, spacing=spacing, origin=origin)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 654
    StructuredQuadGraphExtras.__init__(self, node_y_and_x, sort=sort)
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 518
    links=StructuredQuadLayoutCython.nodes_at_link(self.shape),
  File "/home/aargenti/eclipse-workspace/landlab/landlab/graph/structured_quad/structured_quad.py", line 88
    nodes_at_link = np.empty((n_links, 2), dtype=int)
571 memory blocks: 74.2 KiB
  File "/home/aargenti/eclipse-workspace/landlab/test_memory_leak.py", line 20
    top_stats = snapshot.statistics('traceback')
  File "/home/aargenti/anaconda3/envs/landlab/lib/python3.10/tracemalloc.py", line 533
    grouped = self._group_by(key_type, cumulative)
  File "/home/aargenti/anaconda3/envs/landlab/lib/python3.10/tracemalloc.py", line 498
    traceback = Traceback(frames)
  File "/home/aargenti/anaconda3/envs/landlab/lib/python3.10/tracemalloc.py", line 193
    self._frames = tuple(reversed(frames))

使用此代码进行初始化：

class RasterModelGrid(
    DiagonalsMixIn, DualUniformRectilinearGraph, ModelGrid, RasterModelGridPlotter
):

    def __init__(
        self,
        shape,
        xy_spacing=1.0,
        xy_of_lower_left=(0.0, 0.0),
        xy_of_reference=(0.0, 0.0),
        xy_axis_name=("x", "y"),
        xy_axis_units="-",
        bc=None,
    ):
        shape = tuple(shape)
        xy_spacing = np.asfarray(np.broadcast_to(xy_spacing, 2))
        self._xy_of_lower_left = tuple(np.asfarray(xy_of_lower_left))

        if shape[0] <= 0 or shape[1] <= 0:
            raise ValueError("number of rows and columns must be positive")

        DualUniformRectilinearGraph.__init__(
            self, shape, spacing=xy_spacing[::-1], origin=self.xy_of_lower_left[::-1]
        )
        ModelGrid.__init__(
            self,
            xy_axis_name=xy_axis_name,
            xy_axis_units=xy_axis_units,
            xy_of_reference=xy_of_reference,
        )

        self._node_status = np.full(
            self.number_of_nodes, NodeStatus.CORE, dtype=np.uint8
        )
        self._node_status[self.perimeter_nodes] = NodeStatus.FIXED_VALUE

        if bc is None:
            bc = {"right": "open", "top": "open", "left": "open", "bottom": "open"}

        if "closed" in bc.values():
            self.set_closed_boundaries_at_grid_edges(*grid_edge_is_closed_from_dict(bc))

        self.looped_node_properties = {}

        # List of looped neighbor cells (all 8 neighbors) for
        # given *cell ids* can be created if requested by the user.
        self._looped_cell_neighbor_list = None

        # List of second ring looped neighbor cells (all 16 neighbors) for
        # given *cell ids* can be created if requested by the user.
        self._looped_second_ring_cell_neighbor_list_created = False

在实例化子对象时，为什么会堆积此内存？ 我知道这不是最小的工作代码，但这不是我的代码，我也不熟悉它。如果您无法分辨出什么问题，您建议您测试什么，以查看问题的来源？

Answer 1

鉴于您在Ubuntu上运行，我建议您使用 chap 。

在您的特定情况下，鉴于您看到了内存的积累，但不一定是崩溃，因此您应该为您的流程收集一个现场核心，如下所示：

echo 0x37 > /proc/pid-of-your-process/coredump_filter
gcore pid-of-your-process

chap core-file-just-created

您到达CHAP提示

describe used %ContainerPythonObject /redirectSuffix containers

当 。您可以在该文件中搜索 rastermodelgrid ，您可能会发现其中的许多，并且每个人都很可能参与某种周期。鉴于Python分配的地址，您可以通过在Chap提示中进行类似的操作来理解它是如何锚定的：

describe allocation address-you-just-found \
   /extend %ContainerPythonObject<- \
   /extend %PyDictKeysObject<- \
   /extend %PyDictValuesArray<- \
   /extend %PythonListItems<- \
   /skipUnfavoredReferences true \
   /commentExtensions true \
   /redirectSuffix RasterModelGridExplanation

如果您仔细查看结果文件，这应该可以理解为什么该特定的 rastermodelgridexplanation 仍在记忆中，它仍然可以到达，要么只是参与周期，并且尚未收集垃圾。