如何使用自引用和带有槽的类来pickle和unpickle对象?
当该对象通过其属性之一引用自身时,从带有槽的类中腌制该对象的正确方法是什么?这是一个简单的例子,使用我当前的实现,我不确定它是否 100% 正确:
import weakref
import pickle
class my_class(object):
__slots__ = ('an_int', 'ref_to_self', '__weakref__')
def __init__(self):
self.an_int = 42
self.ref_to_self = weakref.WeakKeyDictionary({self: 1})
# How to best write __getstate__ and __setstate__?
def __getstate__(self):
obj_slot_values = dict((k, getattr(self, k)) for k in self.__slots__)
# Conversion to a usual dictionary:
obj_slot_values['ref_to_self'] = dict(obj_slot_values['ref_to_self'])
# Unpicklable weakref object:
del obj_slot_values['__weakref__']
return obj_slot_values
def __setstate__(self, data_dict):
# print data_dict
for (name, value) in data_dict.iteritems():
setattr(self, name, value)
# Conversion of the dict back to a WeakKeyDictionary:
self.ref_to_self = weakref.WeakKeyDictionary(
self.ref_to_self.iteritems())
这可以通过以下方式进行测试:
def test_pickling(obj):
"Pickles obj and unpickles it. Returns the unpickled object"
obj_pickled = pickle.dumps(obj)
obj_unpickled = pickle.loads(obj_pickled)
# Self-references should be kept:
print "OK?", obj_unpickled == obj_unpickled.ref_to_self.keys()[0]
print "OK?", isinstance(obj_unpickled.ref_to_self,
weakref.WeakKeyDictionary)
return obj_unpickled
if __name__ == '__main__':
obj = my_class()
obj_unpickled = test_pickling(obj)
obj_unpickled2 = test_pickling(obj_unpickled)
这是一个正确/稳健的实现吗?如果my_class
继承自具有__slots__
的类,那么__getstate__
和__setstate__
应该如何编写?由于“循环”字典,__setstate__
内部是否存在内存泄漏?
PEP 307 中有一句话让我想知道是否 pickling my_class
对象完全可以以稳健的方式实现:
__getstate__
方法应该返回一个表示对象状态的可选取值,而不引用对象本身。
这与对象本身的引用被腌制的事实是否冲突?
有很多问题:任何评论、评论或建议将不胜感激!
What is a correct way to pickle an object from a class with slots, when this object references itself through one of its attributes? Here is a simple example, with my current implementation, which I'm not sure is 100 % correct:
import weakref
import pickle
class my_class(object):
__slots__ = ('an_int', 'ref_to_self', '__weakref__')
def __init__(self):
self.an_int = 42
self.ref_to_self = weakref.WeakKeyDictionary({self: 1})
# How to best write __getstate__ and __setstate__?
def __getstate__(self):
obj_slot_values = dict((k, getattr(self, k)) for k in self.__slots__)
# Conversion to a usual dictionary:
obj_slot_values['ref_to_self'] = dict(obj_slot_values['ref_to_self'])
# Unpicklable weakref object:
del obj_slot_values['__weakref__']
return obj_slot_values
def __setstate__(self, data_dict):
# print data_dict
for (name, value) in data_dict.iteritems():
setattr(self, name, value)
# Conversion of the dict back to a WeakKeyDictionary:
self.ref_to_self = weakref.WeakKeyDictionary(
self.ref_to_self.iteritems())
This can be tested for instance with:
def test_pickling(obj):
"Pickles obj and unpickles it. Returns the unpickled object"
obj_pickled = pickle.dumps(obj)
obj_unpickled = pickle.loads(obj_pickled)
# Self-references should be kept:
print "OK?", obj_unpickled == obj_unpickled.ref_to_self.keys()[0]
print "OK?", isinstance(obj_unpickled.ref_to_self,
weakref.WeakKeyDictionary)
return obj_unpickled
if __name__ == '__main__':
obj = my_class()
obj_unpickled = test_pickling(obj)
obj_unpickled2 = test_pickling(obj_unpickled)
Is this a correct/robust implementation? how should __getstate__
and __setstate__
be written if my_class
inherited from a class with __slots__
? is there a memory leak inside __setstate__
because of the "circular" dict?
There is a remark in PEP 307 that makes me wonder whether pickling my_class
objects is at all possible in a robust way:
The
__getstate__
method should return a picklable value representing the object's state without referencing the object itself.
Does this clash with the fact that a reference to the object itself is pickled?
That's a lot of questions: any remark, comment, or advice would be much appreciated!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
看起来原来的帖子建议的效果足够好。
至于 PEP 307 的内容:
我知道这仅意味着 __getstate__ 方法必须返回一个不指向(不可拾取的)原始对象的表示形式。因此,只要不引用原始(不可pickle)对象,返回引用自身的对象就可以了。
It looks like what the original post suggests works well enough.
As for what PEP 307 reads:
I understand that it only means that the
__getstate__
method simply must return a representation that does not point to the (unpickleable) original object. Thus, returning an object that references itself is fine, as long as no reference to the original (unpickleable) object is made.