如何在Python中可靠地进行清理?

发布于 2024-07-25 06:40:24 字数 1068 浏览 10 评论 0原文

我有一些 ctypes 绑定,对于每个 body.New 我应该调用 body.Free。 我绑定的库没有与其余代码隔离的分配例程(它们可以在任何地方调用),并且要使用一些有用的功能,我需要进行循环引用。

我认为如果我能找到一种可靠的方法将析构函数挂接到对象上就可以解决。 (如果他们在数据被删除之前给我回调,weakrefs会有所帮助。

所以显然,当我放入velocity_func时,这段代码会失败:

class Body(object):
    def __init__(self, mass, inertia):
        self._body = body.New(mass, inertia)

    def __del__(self):
        print '__del__ %r' % self
        if body:
            body.Free(self._body)

    ...        

    def set_velocity_func(self, func):
        self._body.contents.velocity_func = ctypes_wrapping(func)

我也尝试通过weakrefs来解决它,这些事情似乎变得更糟,只是在很大程度上更加不可预测

即使我不放入velocity_func,至少当我这样做时会出现循环:

class Toy(object):
    def __init__(self, body):
        self.body.owner = self

...

def collision(a, b, contacts):
    whatever(a.body.owner)

那么如何确保结构将被垃圾收集,即使它们被分配了。 /被共享库释放?

如果您对更多详细信息感兴趣,有存储库:http://bitbucket。 org/cheery/ctypes-chipmunk/

I have some ctypes bindings, and for each body.New I should call body.Free. The library I'm binding doesn't have allocation routines insulated out from the rest of the code (they can be called about anywhere there), and to use couple of useful features I need to make cyclic references.

I think It'd solve if I'd find a reliable way to hook destructor to an object. (weakrefs would help if they'd give me the callback just before the data is dropped.

So obviously this code megafails when I put in velocity_func:

class Body(object):
    def __init__(self, mass, inertia):
        self._body = body.New(mass, inertia)

    def __del__(self):
        print '__del__ %r' % self
        if body:
            body.Free(self._body)

    ...        

    def set_velocity_func(self, func):
        self._body.contents.velocity_func = ctypes_wrapping(func)

I also tried to solve it through weakrefs, with those the things seem getting just worse, just only largely more unpredictable.

Even if I don't put in the velocity_func, there will appear cycles at least then when I do this:

class Toy(object):
    def __init__(self, body):
        self.body.owner = self

...

def collision(a, b, contacts):
    whatever(a.body.owner)

So how to make sure Structures will get garbage collected, even if they are allocated/freed by the shared library?

There's repository if you are interested about more details: http://bitbucket.org/cheery/ctypes-chipmunk/

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

穿越时光隧道 2024-08-01 06:40:24

不幸的是,你想要做的是创建一个对象来分配东西,然后当该对象不再使用时自动释放,这在Python中几乎是不可能的。 del 语句不保证被调用,因此您不能依赖它。

Python 中的标准方法很简单:

try:
    allocate()
    dostuff()
finally:
    cleanup()

或者从 2.5 开始,您还可以创建上下文管理器并使用 with 语句,这是一种更简洁的方法。

但这两者主要适用于在代码片段开头分配/锁定的情况。 如果您想在程序的整个运行过程中分配资源,则需要在启动时、程序的主代码运行之前分配资源,然后再释放。 这里没有介绍一种情况,那就是当您想要动态分配和释放许多资源并在代码中的许多地方使用它们时。 例如,您想要一个内存缓冲区池或类似的池。 但大多数情况都是针对内存的,Python 会为你处理这些问题,所以你不必担心这些。 当然,在某些情况下,您希望对非内存的内容进行动态池分配,然后您需要在示例中尝试的释放类型,而这对于 Python 来说是很棘手的。

What you want to do, that is create an object that allocates things and then deallocates automatically when the object is no longer in use, is almost impossible in Python, unfortunately. The del statement is not guaranteed to be called, so you can't rely on that.

The standard way in Python is simply:

try:
    allocate()
    dostuff()
finally:
    cleanup()

Or since 2.5 you can also create context-managers and use the with statement, which is a neater way of doing that.

But both of these are primarily for when you allocate/lock in the beginning of a code snippet. If you want to have things allocated for the whole run of the program, you need to allocate the resource at startup, before the main code of the program runs, and deallocate afterwards. There is one situation which isn't covered here, and that is when you want to allocate and deallocate many resources dynamically and use them in many places in the code. For example of you want a pool of memory buffers or similar. But most of those cases are for memory, which Python will handle for you, so you don't have to bother about those. There are of course cases where you want to have dynamic pool allocation of things that are NOT memory, and then you would want the type of deallocation you try in your example, and that is tricky to do with Python.

夏末 2024-08-01 06:40:24

如果弱引用没有被破坏,我想这可能会起作用:

from weakref import ref

pointers = set()

class Pointer(object):
    def __init__(self, cfun, ptr):
        pointers.add(self)
        self.ref = ref(ptr, self.cleanup)
        self.data = cast(ptr, c_void_p).value # python cast it so smart, but it can't be smarter than this.
        self.cfun = cfun

    def cleanup(self, obj):
        print 'cleanup 0x%x' % self.data
        self.cfun(self.data)
        pointers.remove(self)

def cleanup(cfun, ptr):
    Pointer(cfun, ptr)

我还尝试过。 重要的是,除了整数之外,指针没有任何对外部指针的强引用。 如果 ctypes 没有释放我应该通过绑定释放的内存,这应该可以工作。 是的,这基本上是一种黑客攻击,但我认为它可能比我之前尝试过的方法效果更好。

编辑:尝试过,在对我的代码进行小幅微调后它似乎可以工作。 令人惊讶的是,即使我从所有结构中删除 del ,它似乎仍然失败。 有趣但令人沮丧。

两者都不起作用,由于一些奇怪的机会,我能够在某些地方删除循环引用,但事情仍然失败。

编辑:好吧..弱引用毕竟被破坏了! 因此,除了强制显式清理之外,Python 中可能没有可靠的清理解决方案。

If weakrefs aren't broken, I guess this may work:

from weakref import ref

pointers = set()

class Pointer(object):
    def __init__(self, cfun, ptr):
        pointers.add(self)
        self.ref = ref(ptr, self.cleanup)
        self.data = cast(ptr, c_void_p).value # python cast it so smart, but it can't be smarter than this.
        self.cfun = cfun

    def cleanup(self, obj):
        print 'cleanup 0x%x' % self.data
        self.cfun(self.data)
        pointers.remove(self)

def cleanup(cfun, ptr):
    Pointer(cfun, ptr)

I yet try it. The important piece is that the Pointer doesn't have any strong references to the foreign pointer, except an integer. This should work if ctypes doesn't free memory that I should free with the bindings. Yeah, it's basicly a hack, but I think it may work better than the earlier things I've been trying.

Edit: Tried it, and it seem to work after small finetuning my code. A surprising thing is that even if I got del out from all of my structures, it seem to still fail. Interesting but frustrating.

Neither works, from some weird chance I've been able to drop away cyclic references in places, but things stay broke.

Edit: Well.. weakrefs WERE broken after all! so there's likely no solution for reliable cleanup in python, except than forcing it being explicit.

氛圍 2024-08-01 06:40:24

在 CPython 中,__del__ 对象的可靠析构函数,因为当引用计数达到零时,它总是会被调用(注意:可能存在这种情况 - 比如循环引用定义了 __del__ 方法的项目 - 引用计数永远不会达到零,但这是另一个问题)。

更新
从评论中,我了解到问题与对象的销毁顺序有关:body是一个全局对象,它在所有其他对象之前被销毁,因此它们不再可用。
实际上,使用全局对象并不好; 不仅是因为这样的问题,还因为维护。

然后,我会用类似这样的内容更改您的类

class Body(object):
    def __init__(self, mass, inertia):
        self._bodyref = body
        self._body = body.New(mass, inertia)

    def __del__(self):
        print '__del__ %r' % self
        if body:
            body.Free(self._body)

...        

def set_velocity_func(self, func):
    self._body.contents.velocity_func = ctypes_wrapping(func)

几个注意事项:

  1. 更改只是添加对全局 body 对象的引用,因此该对象至少会与从该对象派生的所有对象一样存在班级。
  2. 尽管如此,由于单元测试和维护的原因,使用全局对象并不好; 更好的是拥有一个对象工厂,它将为类设置正确的“主体”,并且在单元测试的情况下将轻松放置一个模拟对象。 但这实际上取决于您以及您认为在这个项目中付出多少努力才有意义。

In CPython, __del__ is a reliable destructor of an object, because it will always be called when the reference count reaches zero (note: there may be cases - like circular references of items with __del__ method defined - where the reference count will never reaches zero, but that is another issue).

Update
From the comments, I understand the problem is related to the order of destruction of objects: body is a global object, and it is being destroyed before all other objects, thus it is no longer available to them.
Actually, using global objects is not good; not only because of issues like this one, but also because of maintenance.

I would then change your class with something like this

class Body(object):
    def __init__(self, mass, inertia):
        self._bodyref = body
        self._body = body.New(mass, inertia)

    def __del__(self):
        print '__del__ %r' % self
        if body:
            body.Free(self._body)

...        

def set_velocity_func(self, func):
    self._body.contents.velocity_func = ctypes_wrapping(func)

A couple of notes:

  1. The change is only adding a reference to the global body object, that thus will live at least as much as all the objects derived from that class.
  2. Still, using a global object is not good because of unit testing and maintenance; better would be to have a factory for the object, that will set the correct "body" to the class, and in case of unit test will easily put a mock object. But that's really up to you and how much effort you think makes sense in this project.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文