将标记化单词列表与一组单词进行比较

Question

我想知道评论是否与主题相关，所以我建立了一套与主题相关的词汇

effi_set = {"reminders","medication", "Alarm"
"diet", "carbohydrate","nutrition","weight","IBM", "sport", "activity", "fitbit","blood","insulin",
"Hb1ac" , "data exportation","feedback", "monitoring","recording ","monitor", "record",
"passwords","security","backup","protection",
"information","education","complication","risk","prevent","contact","consultation",
"facebook","twitter","social media","mail","FAQ","doctor",
"data","offline","language","location","region","country",
"devise","glucometer","bluetooth","automation","carb","barcode","food","syncronize","PHR","import"}

我将每个评论标记化，以将标记化的单词与主题集进行比较

for line in df["content"]:
    tokenized_words =word_tokenize(line)
    for item in tokenized_words:
        if item not in effi_set:
            df["efficient"] = False
        else:
            df["efficient"] = True

结果是所有的评论都是假的，事实并非如此

Answer 1

df["efficient"] = False正在填充整个列

您必须一次修改一行

df["efficient"] = False
for index, line in df["content"].iteritems():
    tokenized_words =word_tokenize(line)
    for item in tokenized_words:
        if item in effi_set:
            df.at[index, "efficient"] = True
            continue