我正在尝试从磁盘将自定义管道作为spacy 3.0中的管道加载为自定义预训练模型:
工厂代码如下:
@CustomEng.factory("ner-crf")
def create_my_component(nlp, name):
crf_extractor = CRFExtractor().from_disk("path-to-model")
return CRFEntityExtractor(nlp, crf_extractor=crf_extractor)
然后我在我的语言课上添加了“ner crf”,如下所示:
nlp = spacy.blank('custom-eng')
nlp.add_pipe('ner-crf')
nlp.to_disk('../model')
我认为有一件事可能与此相关:当我使用to_disk
来保存nlp
对象时,保存的对象中没有ner-crf
包(我刚才添加的管道)
然后运行此CLI命令以评估NER管道:
python -m spacy evaluate ../model/ ../corpus/dev.spacy --output ../model/metrics.json --gpu-id 0 --code ../../../spacy_utils/custom-eng/__init__.py
但我得到了这个错误:
Traceback (most recent call last):
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
setup_cli()
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/_util.py", line 69, in setup_cli
command(prog_name=COMMAND)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
return callback(**use_params) # type: ignore
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/evaluate.py", line 42, in evaluate_cli
evaluate(
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/evaluate.py", line 75, in evaluate
nlp = util.load_model(model)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py", line 326, in load_model
return load_model_from_path(Path(name), **kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py", line 392, in load_model_from_path
return nlp.from_disk(model_path, exclude=exclude)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/language.py", line 1883, in from_disk
util.from_disk(path, deserializers, exclude)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py", line 1176, in from_disk
reader(path / key)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/language.py", line 1877, in <lambda>
deserializers[name] = lambda p, proc=proc: proc.from_disk(
TypeError: from_disk() got an unexpected keyword argument 'exclude'
我使用的自定义NER类属于spacy-crfsuite库,该库在spacy 2中运行良好,但它们还没有spacy 3的示例代码,因此我正在尝试让它在spacy 3.0中运行
从SPACYV3.0开始,管道组件在其
to_disk
方法上支持exclude
关键字。您可以将exclude
关键字添加到函数中,给它一个默认值,并且不在函数体中使用它的值,这个错误应该得到解决为了完整起见,这里是从v2到v3的迁移指南,其中可能包括一些其他有趣的指针:https://spacy.io/usage/v3#migrating
相关问题 更多 >
编程相关推荐