无法导入transformers.models.auto.tokenization_auto MacOS

Question

我正在尝试使用Jupyter Notebook从变形金刚库中加载汽车模型，自动源。

from transformers import AutoModel, AutoTokenizer

这会导致以下错误（提供了完整的StackTrace），

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/utils/import_utils.py:933, in _LazyModule._get_module(self, module_name)
    932 try:
--> 933     return importlib.import_module("." + module_name, self.__name__)
    934 except Exception as e:

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1030, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1007, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:986, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:680, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:850, in exec_module(self, module)

File <frozen importlib._bootstrap>:228, in _call_with_frames_removed(f, *args, **kwds)

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py:27, in <module>
     26 from ...tokenization_utils_base import TOKENIZER_CONFIG_FILE
---> 27 from ...tokenization_utils_fast import PreTrainedTokenizerFast
     28 from ...utils import get_file_from_repo, is_sentencepiece_available, is_tokenizers_available, logging

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py:24, in <module>
     22 from typing import Any, Dict, List, Optional, Tuple, Union
---> 24 import tokenizers.pre_tokenizers as pre_tokenizers_fast
     25 from tokenizers import Encoding as EncodingFast

ModuleNotFoundError: No module named 'tokenizers'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 from transformers import AutoModel, AutoTokenizer

File <frozen importlib._bootstrap>:1055, in _handle_fromlist(module, fromlist, import_, recursive)

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/utils/import_utils.py:924, in _LazyModule.__getattr__(self, name)
    922 elif name in self._class_to_module.keys():
    923     module = self._get_module(self._class_to_module[name])
--> 924     value = getattr(module, name)
    925 else:
    926     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/utils/import_utils.py:923, in _LazyModule.__getattr__(self, name)
    921     value = self._get_module(name)
    922 elif name in self._class_to_module.keys():
--> 923     module = self._get_module(self._class_to_module[name])
    924     value = getattr(module, name)
    925 else:

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/transformers/utils/import_utils.py:935, in _LazyModule._get_module(self, module_name)
    933     return importlib.import_module("." + module_name, self.__name__)
    934 except Exception as e:
--> 935     raise RuntimeError(
    936         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
    937         f" traceback):\n{e}"
    938     ) from e

RuntimeError: Failed to import transformers.models.auto.tokenization_auto because of the following error (look up to see its traceback):
No module named 'tokenizers'

我正在使用MacOS Monterey v12.4。我已经创建了张量流的虚拟环境，并且还安装了令牌和变压器。我尝试再次尝试进行插件，但已经安装了。

% pip install tokenizers
Requirement already satisfied: tokenizers in /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/tokenizers-0.12.1.dev0-py3.9-macosx-11.0-arm64.egg (0.12.1.dev0)

有人可以启发我关于问题是什么以及如何解决这个问题吗？我已经检查了互联网是否有可能，但找不到任何东西。我是新手，所以如果这是一个简单的问题，请原谅我。请随时问我更多解决这个问题所需的东西。