当我试图在aws ubuntu实例中运行以下pickle程序时,我收到了“killed”消息,它没有生成pickle。不过,当我在本地机器(Mac)上尝试时,同样的方法也可以。我使用python3运行程序:
import nltk
import random
from nltk.classify.scikitlearn import SklearnClassifier
import pickle
from nltk.tokenize import word_tokenize
documents_f = open("documents.pickle", "rb")
documents = pickle.load(documents_f)
documents_f.close()
word_features5k_f = open("word_features5k.pickle", "rb")
word_features = pickle.load(word_features5k_f)
word_features5k_f.close()
def find_features(document):
words = word_tokenize(document)
features = {}
for w in word_features:
features[w] = (w in words)
#print (features)
return features
featuresets = [(find_features(rev), category) for (rev, category) in documents]
save_featuresets = open("featuresets.pickle","wb")
pickle.dump(featuresets, save_featuresets)
save_featuresets.close()
我相信这可能是由于内存问题,因为我使用的是aws免费轮胎。有人请告诉我怎么解决这个问题?在
发行日期:
^{pr2}$
您的EC2实例内存不足,进程被the OOM killer终止。在
升级到更大的实例类型,或者找到一种方法使进程使用更少的内存。(例如,您可能需要考虑一次处理一个文档,而不是一次加载所有文档。)
相关问题 更多 >
编程相关推荐