无法将对象放入队列

Question

我想将 scapy.layers.dhcp.BOOTP 的实例放在 multiprocessing.Queue 上。每次我调用 put() 时都会发生以下异常：

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 242, in _feed
    send(obj)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

当然，尝试直接使用 pickle.dumps() 来 pickle 实例也会失败。但为什么这个类不能 picklable 呢？

对于所有没有安装 scapy 的人：

class BOOTP(Packet):
    name = "BOOTP"
    fields_desc = [ ByteEnumField("op",1, {1:"BOOTREQUEST", 2:"BOOTREPLY"}),
                    ByteField("htype",1),
                    ByteField("hlen",6),
                    ByteField("hops",0),
                    IntField("xid",0),
                    ShortField("secs",0),
                    FlagsField("flags", 0, 16, "???????????????B"),
                    IPField("ciaddr","0.0.0.0"),
                    IPField("yiaddr","0.0.0.0"),
                    IPField("siaddr","0.0.0.0"),
                    IPField("giaddr","0.0.0.0"),
                    Field("chaddr","", "16s"),
                    Field("sname","","64s"),
                    Field("file","","128s"),
                    StrField("options","") ]
    def guess_payload_class(self, payload):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            return DHCP
        else:
            return Packet.guess_payload_class(self, payload)
    def extract_padding(self,s):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            # set BOOTP options to DHCP magic cookie and make rest a payload of DHCP options
            payload = self.options[len(dhcpmagic):]
            self.options = self.options[:len(dhcpmagic)]
            return payload, None
        else:
            return "", None
    def hashret(self):
        return struct.pack("L", self.xid)
    def answers(self, other):
        if not isinstance(other, BOOTP):
            return 0
        return self.xid == other.xid

还有其他方法可以将此实例“传输”到另一个子进程吗？

Answer 1

嗯，问题是你不能 pickle 函数类型。这就是您执行 type(some_user_function) 时得到的结果。请参阅：

>>> import types
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'picke' is not defined
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python26\lib\pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "C:\Python26\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <type 'function'>: it's not found as __built
n__.function

因此，这样的函数类型存储在您尝试发送的对象上的某个位置。它不在您粘贴的代码中，所以我猜它在超类中。

也许您可以简单地发送创建 scapy.layers.dhcp.BOOTP 实例而不是实例所需的所有参数来避免问题？

Answer 2

可能有助于诊断此类问题的另一件事是使用pickle模块而不是cPickle（它必须由queues.py隐式使用）

我也遇到过类似的情况，得到一条完全无用的消息，

Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我走进了调试器，发现正在腌制的对象，并尝试将其传递给

pickle.dump(myobj,open('outfile','w'),-1)  

并得到更有帮助的：

PicklingError: Can't pickle <function findAllRefs at 0x105809f50>: 
   it's not found as buildsys.repoclient.findAllRefs

这更直接地指出了有问题的代码。

Answer 3

我使用的解决方案是对数据包进行str处理，然后将其放入队列中......