具有混合列类型的 Pandas DataFrame 的带外（零拷贝）酸洗

Question

我正在尝试在从单独的控制台会话启动的进程之间实现 Pandas DataFrame 的零拷贝共享。请考虑以下两个Python文件：

生产者.py：

import pandas as pd 
import numpy as np 
import pickle

df = pd.DataFrame({'text': ['a','b','c'], 'ints':[1,2,3], 'floats': [1.0,2,3]})


print(df)
#   text  ints  floats
# 0    a     1     1.0
# 1    b     2     2.0
# 2    c     3     3.0

# prints as expected!


buffers = []

with open("my_df.pickle", "wb") as f:
    pickle.dump(df, protocol=5, buffer_callback=buffers.append, file=f)


for b in buffers:
    print(len(b.raw()))
# 24
# 24

# only prints 2 buffers! Expected 3 buffers (1 for each column)

随后我从另一个控制台运行consumer.py ：

import pandas as pd 
import numpy as np 
import pickle


buffers = [pickle.PickleBuffer(bytes(24)), pickle.PickleBuffer(bytes(24))]

f = open("my_df.pickle", "rb")
df = pickle.load(f, buffers=buffers)


print(df)
#   text  ints  floats
# 0    a     0     0.0
# 1    b     0     0.0
# 2    c     0     0.0

# Unexpected output. Numerical values are zero'd. And only 1 out of 3 columns ('text') is correctly populated. 

似乎 2 个 PickleBuffer 仅适用于数字列，但它们没有正确显示，而文本列却正确！

（显然，目的是正确地显示完整的 DataFrame。）

非常欢迎任何建议！