mycorpus.txt
Human where's machine interface for lab abc computer applications
A where's survey of user opinion of computer system response time
stopwords.txt
let's
ain't
there's
以下代码
corpus = set()
for line in open("path\\to\\mycorpus.txt"):
corpus.update(set(line.lower().split()))
print corpus
stoplist = set()
for line in open("C:\\Users\\Pankaj\\Desktop\\BTP\\stopwords_new.txt"):
stoplist.add(line.lower().strip())
print stoplist
给出以下输出
set(['a', "where's", 'abc', 'for', 'of', 'system', 'lab', 'machine', 'applications', 'computer', 'survey', 'user', 'human', 'time', 'interface', 'opinion', 'response'])
set(['let\x92s', 'ain\x92t', 'there\x92s'])
为什么在第二组中撇号变成了\x92??
窗口1252编码中的代码点92(十六进制)是Unicode代码点2019(十六进制),它是“右单引号”。这看起来非常像一个撇号,很可能是您在
stopwords.txt
中的实际字符,我从python在中的解释方式猜到,它是在windows-1252中编码的,或者是共享ASCII和’
代码点值的编码。'与'
相关问题 更多 >
编程相关推荐