attributeError：can can get get属性＆＃x27; my_func＆＃x27; ON＆lt;模块＆＃x27; __ main __＆＃x27;从main.py＆＃x27;＆gt;

Question

我想根据笔记本创建Python脚本，以使用相同的.pkl文件获得运行时。

---

在这一行：

learn = load_learner('model.pkl', cpu=True)

我得到此错误：

(project) me@ubuntu-pcs:~/PycharmProjects/project$ python main.py 
Traceback (most recent call last):
  File "main.py", line 6, in <module>
    from src.train.train_model import train
  File "/home/me/PycharmProjects/project/src/train/train_model.py", line 17, in <module>
    learn = load_learner('yasmine-sftp/export_2.pkl', cpu=True)  # to run on GPU
  File "/home/me/miniconda3/envs/project/lib/python3.6/site-packages/fastai/learner.py", line 384, in load_learner
    res = torch.load(fname, map_location='cpu' if cpu else None, pickle_module=pickle_module)
  File "/home/me/miniconda3/envs/project/lib/python3.6/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/me/miniconda3/envs/project/lib/python3.6/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/me/miniconda3/envs/project/lib/python3.6/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'Tf' on <module '__main__' from 'main.py'>

这是因为要打开.pkl文件，我需要用于训练它的原始功能。

值得庆幸的是，回头看笔记本，tf（o）是否存在：

def Tf(o):
    return '/mnt/scratch2/DLinTHDP/PathLAKE/Version_4_fastai/Dataset/CD8/Train/masks/'+f'{o.stem}_P{o.suffix}'

但是，我在我的Python脚本中放置tf（o）的任何地方，我仍然会遇到相同的错误。

在错误消息中，我应该在哪里放置tf（o）？：＆lt; module'__ -main__'来自'main..py'＆gt;建议将其放入main（）或下方，如果__name__ ...。

我到处都尝试过。导入tf（o）也行不通。

---

python脚本

main.py：

import glob
from pathlib import Path

from train_model import train

ROOT = Path("folder/path")  # Detection Folder


def main(root: Path):
    train(root)


if __name__ == '__main__':
    main(ROOT)

train_model.py.py：

from pathlib import Path

from fastai.vision.all import *


folder_path = Path('.')

learn = load_learner('model.pkl', cpu=True)  # AttributeError
learn.load('model_3C_34_CELW_V_1.1')  # weights


def train(root: Path):
    # ...

---

我无法检查文件：

(project) me@ubuntu-pcs:~/PycharmProjects/project$ python -m pickletools -a model.pkl
Traceback (most recent call last):
  File "/home/me/miniconda3/envs/project/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/me/miniconda3/envs/project/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/me/miniconda3/envs/project/lib/python3.6/pickletools.py", line 2830, in <module>
    args.indentlevel, annotate)
  File "/home/me/miniconda3/envs/project/lib/python3.6/pickletools.py", line 2394, in dis
    for opcode, arg, pos in genops(pickle):
  File "/home/me/miniconda3/envs/project/lib/python3.6/pickletools.py", line 2242, in _genops
    arg = opcode.arg.reader(data)
  File "/home/me/miniconda3/envs/project/lib/python3.6/pickletools.py", line 373, in read_stringnl_noescape
    return read_stringnl(f, stripquotes=False)
  File "/home/me/miniconda3/envs/project/lib/python3.6/pickletools.py", line 359, in read_stringnl
    data = codecs.escape_decode(data)[0].decode("ascii")
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 63: ordinal not in range(128)

Answer 1

根据Ecatkins Edward Atkins的说法
https：// forums。 fast.ai/t/error-loading-saved-model-with-custom-loss-function/37627/7

Load_learner在原始模块中未找到用于定义模型的类时失败。因此，覆盖Pickle类加载程序（FIND_CLASS）以在指定的模块中搜索。

import imp, sys
pickle2 = imp.load_module('pickle2', *imp.find_module('pickle'))

# The module where the class is now found.
MODULE = "MY.MODULE"

class CustomUnpickler(pickle2.Unpickler):

    def find_class(self, module, name):
        try:
            return super().find_class(module, name)
        except AttributeError:
            if module == "__main__":
                print(f"load_learner can't find {name} in original module {module}; getting it from {MODULE}")
                module = MODULE
                return super().find_class(module, name)


# Modified load_learner from fastai.basic_train.py (fastai==1.0.61), according to ecatkins Edward Atkins
# https://forums.fast.ai/t/error-loading-saved-model-with-custom-loss-function/37627/7
def load_learner2(path:PathOrStr, file:PathLikeOrBinaryStream='export.pkl', test:ItemList=None, tfm_y=None, **db_kwargs):
    "Load a `Learner` object saved with `export_state` in `path/file` with empty data, optionally add `test` and load on `cpu`. `file` can be file-like (file or buffer)"
    source = Path(path)/file if is_pathlike(file) else file
    # state = torch.load(source, map_location='cpu') if defaults.device == torch.device('cpu') else torch.load(source)
    # Use custom class loader here
    pickle2.Unpickler = CustomUnpickler
    state = torch.load(source, map_location='cpu',  pickle_module=pickle2) if defaults.device == torch.device('cpu') else torch.load(source,  pickle_module=pickle2)
    model = state.pop('model')
    src = LabelLists.load_state(path, state.pop('data'))
    if test is not None: src.add_test(test, tfm_y=tfm_y)
    data = src.databunch(**db_kwargs)
    cb_state = state.pop('cb_state')
    clas_func = state.pop('cls')
    res = clas_func(data, model, **state)
    res.callback_fns = state['callback_fns'] #to avoid duplicates
    res.callbacks = [load_callback(c,s, res) for c,s in cb_state.items()]
    return res

Answer 2

问题

为什么我遇到该错误是BC tf（）函数用于训练model.pkl文件，在相同的它是在笔记本文件中完成的）。

此文章

泡菜是懒惰的，不会序列化类定义
定义。相反，它保存了如何找到课程的参考
（它居住的模块及其名称）

解决方案

有一个称为dill的扩展名为dill，它确实会序列化python对象和函数等。 ：％7E：文本=约％20Dill，至％20A％20Python％20object％20 hirarchy。” rel =“ nofollow noreferrer”> pypi

Answer 3

另一种方法是手动复制培训环境的命名空间。

就我而言

global Tf
Tf = None