<p>我正在尝试使用transformers包中的BertTokenizer部分。首先,我安装如下</p>
<pre><code>pip install transformers
</code></pre>
<p>这说明它成功了</p>
<p>当我尝试导入包的以下部分时,我得到以下结果</p>
<pre><code> from transformers import BertTokenizer
Traceback (most recent call last):
File "<ipython-input-2-89505a24ece6>", line 1, in <module>
from transformers import BertTokenizer
File "C:\Users\User\anaconda3\lib\site-packages\transformers\__init__.py", line 22, in <module>
from .integrations import ( # isort:skip
File "C:\Users\User\anaconda3\lib\site-packages\transformers\integrations.py", line 42, in <module>
from .trainer_utils import PREFIX_CHECKPOINT_DIR, BestRun # isort:skip
File "C:\Users\User\anaconda3\lib\site-packages\transformers\trainer_utils.py", line 10, in <module>
from .tokenization_utils_base import ExplicitEnum
File "C:\Users\User\anaconda3\lib\site-packages\transformers\tokenization_utils_base.py", line 31, in <module>
from tokenizers import AddedToken
File "C:\Users\User\anaconda3\lib\site-packages\tokenizers\__init__.py", line 17, in <module>
from .tokenizers import Tokenizer, Encoding, AddedToken
ModuleNotFoundError: No module named 'tokenizers.tokenizers'
</code></pre>
<p>这个包在这里很详细,所以我想应该可以买到
<a href="https://huggingface.co/transformers/model_doc/bert.html" rel="nofollow noreferrer">https://huggingface.co/transformers/model_doc/bert.html</a></p>