遍历对象层次结构pickle样式
我需要对在发生之前被腌制的对象进行某种处理。更准确地说,对于某个基类的子类的实例,我希望对完全不同的东西进行腌制,然后在加载时重新创建。
我知道 __getstate__
& __setstate__
然而,这是一种非常侵入性的方法。我的理解是,这些是私有方法(以双下划线开头:__
),因此会受到名称修改的影响。因此,这实际上会迫使我为我想要受到这种非标准行为影响的每个类重新定义这两种方法。此外,我并没有真正完全控制所有类的层次结构。
我想知道是否有某种简短的方法可以挂钩 pickle 过程并应用 __getstate__ 和 __setstate__ 提供的这种控制,但无需将 pickled 类修改为这样的。
为好奇的人提供一个旁注。这是取自使用 Django 和 Celery 的项目的用例。 Django 模型要么不可pickle,要么非常不实用且pickle 很麻烦。因此,更建议腌制 ID + 模型类的值对。然而,有时并不是直接腌制模型,而是模型字典、模型列表、模型列表列表等等。这迫使我编写大量我非常不喜欢的复制粘贴代码。对 pickling 模型的需求本身来自 Django-celery 设置,其中函数及其调用参数被安排为稍后执行。不幸的是,在这些争论中,通常有很多模型混合在一些不平凡的层次结构中。
编辑
我确实有可能指定 Celery 使用的自定义序列化器,所以这实际上是一个能够在 pickle 上构建一个稍微修改的序列化器而不需要太多努力的问题。
I'm in a need for doing some sort of processing on the objects that get pickled just before it happens. More precisely for instances of subclasses of a certain base class I would like something totally different to be pickled instead and then recreated on loading.
I'm aware of __getstate__
& __setstate__
however this is a very invasive approach. My understanding is that these are private methods (begin with double underscore: __
), and as such are subject to name mangling. Therefore this effectively would force me to redefine those two methods for every single class that I want to be subject to this non standard behavior. In addition I don't really have a full control over the hierarchy of all classes.
I was wondering if there is some sort of brief way of hooking into pickling process and applying this sort of control that __getstate__
and __setstate__
give but without having to modify the pickled classes as such.
A side note for the curious ones. This is a use case taken from a project using Django and Celery. Django models are either unpickable or very unpractical and cumbersome to pickle. Therefore it's much more advisable to pickle pairs of values ID + model class instead. However sometimes it's not the model directly that is pickled but rather a dictionary of models, a list of models, a list of lists of models, you name it. This forces me to write a lot of copy-paste code that I really dislike. A need for pickling models comes itself from Django-celery setup, where functions along with their call arguments are scheduled for later execution. Unfortunately among those arguments there are usually a lot of models mixed up in some nontrivial hierarchy.
EDIT
I do have a possibility of specifying a custom serializer to be used by Celery, so it's really a question of being able to build a slightly modified serializer on top of pickle without much effort.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
唯一相关的附加钩子是 reduce() 和 __reduce__ex()
http://docs.python.org/library/pickle.html
__reduce__ 和 __reduce_ex__ 有什么区别?
Python:确保我的类仅使用最新协议进行腌制
不确定它们是否真的提供了您特别需要的内容。
The only additional hooks that are related are reduce() and __reduce__ex()
http://docs.python.org/library/pickle.html
What is the difference between __reduce__ and __reduce_ex__?
Python: Ensuring my class gets pickled only with the latest protocol
Not sure if they really provide what you need in particular.