UnicodeEncodeError:Ascii编解码器无法对位置0中的字符u2581进行编码：序号不在范围内（128）

Question

Traceback (most recent call last): File "train.py", line 18, in tf.app.run(main=nmt.main, argv=[os.getcwd() + '\nmt\nmt\nmt.py'] + unparsed) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/nmt.py", line 551, in main run_main(FLAGS, default_hparams, train_fn, inference_fn) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/nmt.py", line 544, in run_main train_fn(hparams, target_session=target_session) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 271, in train sample_tgt_data) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 142, in run_full_eval sample_src_data, sample_tgt_data) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 55, in run_sample_decode infer_model.batch_size_placeholder, summary_writer) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 454, in _sample_decode utils.print_out(b" src: " + utils.format_sentence(src_data[decode_id], hparams.subword_option))
File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/utils/misc_utils.py", line 193, in format_sentence sentence = format_spm_text(sentence) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/utils/misc_utils.py", line 181, in format_spm_text return u"".join(format_text(symbols).decode("utf-8").split()).replace( File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2581' in position 0: ordinal not in range(128)

Traceback (most recent call last): File "train.py", line 18, in tf.app.run(main=nmt.main, argv=[os.getcwd() + '\nmt\nmt\nmt.py'] + unparsed) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/nmt.py", line 551, in main run_main(FLAGS, default_hparams, train_fn, inference_fn) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/nmt.py", line 544, in run_main train_fn(hparams, target_session=target_session) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 271, in train sample_tgt_data) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 142, in run_full_eval sample_src_data, sample_tgt_data) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 55, in run_sample_decode infer_model.batch_size_placeholder, summary_writer) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/train.py", line 454, in _sample_decode utils.print_out(b" src: " + utils.format_sentence(src_data[decode_id], hparams.subword_option))
File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/utils/misc_utils.py", line 193, in format_sentence sentence = format_spm_text(sentence) File "/home/paperspace/Desktop/nmt-chatbot/nmt/nmt/utils/misc_utils.py", line 181, in format_spm_text return u"".join(format_text(symbols).decode("utf-8").split()).replace( File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2581' in position 0: ordinal not in range(128)

我在这几行代码中发现了一个错误：

def format_spm_text(symbols):
  """Decode a text in SPM (https://github.com/google/sentencepiece) 
  format."""
  return u"".join(format_text(symbols).decode("utf-8").split()).replace(
      u"\u2581", u" ").strip().encode("utf-8")

我试图通过运行一个名为'火车.py'. 我使用命令'sudo python火车.py我目前在Ubuntu中的python版本是3.6版本。在我的本地MacOS上，同样的代码似乎运行得很好，但我在上面运行的是python2.7版本。在

Answer 1

试试这个：

def format_spm_text(symbols): 
  return u"".join(format_text(symbols).decode("unicode-escape").split()).replace(
  u"\u2581", u" ").strip().encode("utf-8")