莳萝有时会使物体的属性消失
我有一个类 1 （protpkg），带有自定义__新的__新__方法，该方法使用其他自定义类别的对象 2 2 < /em>（<代码>状态）作为参数，并将其保存为实例属性。然后，此类的初始化（__ INT __） 1 使用Multiprocess pool pool 执行其类别之一（ 1 ）并行（三个呼叫apply_async）。请注意使用多处理而不是多处理，以便使用莳萝，并且类方法腌制没有问题。
在pool内的类（ 1 ）方法的每个执行中，我猜是制作了名称空间的副本，这意味着类的当前实例< em> 1 ，也是传递给它并保存为属性的类 2 的实例，被莳萝腌制和未挑选。但是，有时莳萝似乎无法正确腌制并取消通过的 2 的对象，其属性消失了。因此，当尝试“重建”类 1 对象，并且在初始化期间尝试访问该类 2 对象的属性，则会出现错误由于这些属性已经消失：
Process ForkPoolWorker-3:
Traceback (most recent call last):
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/process.py", line 315, in _bootstrap
self.run()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/pool.py", line 114, in worker
task = get()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/queues.py", line 371, in get
return _ForkingPickler.loads(res)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 327, in loads
return load(file, ignore, **kwds)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 313, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 525, in load
obj = StockUnpickler.load(self)
File "/mnt/c/Users/franx/Desktop/TFM/bug/Pkgs.py", line 42, in __new__
new = super().__new__(cls, state, **kwargs)
File "/mnt/c/Users/franx/Desktop/TFM/bug/Pkgs.py", line 13, in __new__
new._pdbf = new.state._pdbf
AttributeError: 'State' object has no attribute '_pdbf'
我制作了一个带有可重现代码的GitHub回购：一个名为bug的Python软件包，其中包含这些类的代码来重现该错误，也可以进行测试笔记本或测试笔记本脚本： https://github.com/frannerin/bug 。我使用的是Linux CentOS 7.5.1804机器和Miniconda3 4.11.0环境，具有Python 3.9.12和多进程0.70.12.2（带有DILL 0.3.4）。
奇怪的是，要使事情变得更加令人费解，我观察到有时发送到pool的第一个过程正确起作用（类 2 的对象具有其所有属性），而另一个两个失败。另外，如果我放置time.sleep（5）在每次执行apply_async之后，所有过程通常也可以正常工作。
即使您没有提供实际代码的完整解决方案，我也愿意尝试尝试自己的想法。谢谢！
I have a class 1 (Protpkg) with a custom-defined __new__ method that uses an object of a different custom class 2 (State) as argument and saves it as instance attribute. Then, the initialization (__init__) of this class 1 uses a multiprocess Pool to execute one of its class (1) methods in parallel (three calls to apply_async). Notice the use of multiprocess instead of multiprocessing, so that dill is used and there isn't a problem with class method pickling.
In each of the executions of the class (1) method inside the Pool, I guess that a copy of the namespace is made, meaning that the present instance of the class 1, and also the instance of class 2 that is passed to it and gets saved as attribute, are pickled and unpickled by dill. However, sometimes dill seems to fail at correctly pickling and unpickling the object of class 2 that is passed, and its attributes disappear. Thus, when the class 1 object is attempted to be "reconstructed" and the attributes of the class 2 object passed to it are attempted to be accessed during initialization, an Error arises as these attributes have disappeared:
Process ForkPoolWorker-3:
Traceback (most recent call last):
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/process.py", line 315, in _bootstrap
self.run()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/pool.py", line 114, in worker
task = get()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/multiprocess/queues.py", line 371, in get
return _ForkingPickler.loads(res)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 327, in loads
return load(file, ignore, **kwds)
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 313, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/home/fnerin/miniconda3/envs/AlloViz/lib/python3.9/site-packages/dill/_dill.py", line 525, in load
obj = StockUnpickler.load(self)
File "/mnt/c/Users/franx/Desktop/TFM/bug/Pkgs.py", line 42, in __new__
new = super().__new__(cls, state, **kwargs)
File "/mnt/c/Users/franx/Desktop/TFM/bug/Pkgs.py", line 13, in __new__
new._pdbf = new.state._pdbf
AttributeError: 'State' object has no attribute '_pdbf'
I have made a GitHub repo with reproducible code: a Python package named bug that has the code of these classes to reproduce the error, and also either a testing notebook or a testing Python script: https://github.com/frannerin/bug. I am using a Linux CentOS 7.5.1804 machine and a Miniconda3 4.11.0 environment with Python 3.9.12 and multiprocess 0.70.12.2 (with dill 0.3.4).
Strangely, to make things more convoluted, I have observed that sometimes the first process sent to the Pool works correctly (the object of class 2 has all its attributes) but the other two fail. Alternatively, if I place time.sleep(5) after each execution of apply_async, all of the processes usually work correctly too.
Even if you don't provide a complete solution with actual code, I am open to ideas to try and to look into myself. Thanks!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论