Numba JIT模式内存泄漏问题？

发布于 2025-02-12 06:08:19 字数 835 浏览 3 评论 0原文

我目前在使用另一个jitter函数的功能上将JIT装饰器在一个函数上遇到的函数时，我目前面临着一个奇怪的内存泄漏问题，该功能返回产量样式

的想法是设计的想法，

@njit
def main_ ():
    ss = []
    heapq.heapify (ss)
    yd = generator_()
    result = 0
    while 1:
        result = next (yd)
        if result == []:
             break
        heapq.heappush (result)
    return list (ss)

@njit
def generator_():
    for i in range (100000000):
        yd2 = generator_2 (i)
        while 1:
            result = next (yd2)
            if result == []:
                break
            yield [i, result]
    yield []

@njit
def generator_2(ii):
    for i in range (100000000):
        yield ([ii, i*2, i*i])
    yield []

即如果函数为main_具有NJIT或NJIT或JIT删除了，不会发生内存问题。我也称为GC Collection，但似乎AINT做了很多事情。我还发现了一些讨论表明，它将与LLVM相关，因为它存储了它编制的所有内容。

任何人都可以给我一些意见，问题是什么。

感谢您

原文

I am currently facing a weird memory leak problem when having jit decorator on a function which uses heapify takings results from another jitted function which return results in yield style

here is the idea of the design to indicate the issue

@njit
def main_ ():
    ss = []
    heapq.heapify (ss)
    yd = generator_()
    result = 0
    while 1:
        result = next (yd)
        if result == []:
             break
        heapq.heappush (result)
    return list (ss)

@njit
def generator_():
    for i in range (100000000):
        yd2 = generator_2 (i)
        while 1:
            result = next (yd2)
            if result == []:
                break
            yield [i, result]
    yield []

@njit
def generator_2(ii):
    for i in range (100000000):
        yield ([ii, i*2, i*i])
    yield []

If the function main_ having the njit or jit removed, the memory issue would not happened. I have also called gc collection but it seems like aint doing much. I have also found some discussion shows it would be relate to llvm as it store everything it compiled.

Would anyone would be able to give me some opinion what exact the problem is.

Thanks you

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

云之铃。 2025-02-19 06:08:25

我刚刚面对同样的问题

import gc
import numpy as np
import numba


@numba.jit(cache=False)
def main(x):
    x**2


gc.enable(  )
gc.set_debug(gc.DEBUG_SAVEALL)
x = np.random.random((10,))
main(x)
gc.collect()
assert not len(gc.garbage), f"found {len(gc.garbage)} objects."

提出了ostertionerror

I've just faced the same issue

import gc
import numpy as np
import numba


@numba.jit(cache=False)
def main(x):
    x**2


gc.enable(  )
gc.set_debug(gc.DEBUG_SAVEALL)
x = np.random.random((10,))
main(x)
gc.collect()
assert not len(gc.garbage), f"found {len(gc.garbage)} objects."

raises AssertionError

回复收藏 0 原文

~没有更多了~