TypeError：如果未指定评分，则通过的估计员应采用“评分”方法

Question

我使用PyTorch神经网络已经有一段时间了。我决定添加一个排列特性重要性记分器，这开始引起一些问题

我得到“TypeError:如果没有指定评分，通过的估计器应该有一个‘评分’方法。估计器<；class‘skorch.net.NeuralNet’>；[未初始化]( 模块=<；类'main.run..multi-layerPredictor'>；， )没有。“-错误消息。这是我的密码：

class MultiLayerPredictor(torch.nn.Module):
    def __init__(self, input_shape=9152, output_shape=1, hidden_dim=1024, **kwargs):
        super().__init__()
        self.fc1 = torch.nn.Linear(in_features=input_shape, out_features=hidden_dim)
        self.fc2 = torch.nn.Linear(in_features=hidden_dim, out_features=hidden_dim)
        self.fc3 = torch.nn.Linear(in_features=hidden_dim, out_features=output_shape)

    def forward(self, x):
        l1 = torch.relu(self.fc1(x))
        l2 = torch.relu(self.fc2(l1))
        return torch.sigmoid(self.fc3(l2)).reshape(-1)

print("Moving to wrapping the neural net")
net = NeuralNet(
    MultiLayerPredictor,
    criterion=nn.MSELoss,
    max_epochs=10,
    optimizer=optim.Adam,
    lr=0.1,
    iterator_train__shuffle=True
)

print("Moving to finding optimal hyperparameters")

lr = (10**np.random.uniform(-5,-2.5,1000)).tolist()
params = {
    'optimizer__lr': lr,
    'max_epochs':[300,400,500],
    'module__num_units': [14,20,28,36,42],
    'module__drop' : [0,.1,.2,.3,.4]
}

gs = RandomizedSearchCV(net,params,refit=True,cv=3,scoring='neg_mean_squared_error',n_iter=100)
gs.fit(X_train_scaled,y_train);

def report(results, n_top=3):
    for i in range(1, n_top + 1):
        candidates = np.flatnonzero(results['rank_test_score'] == i)
    for candidate in candidates:
        print("Model with rank: {0}".format(i))
        print("Mean validation score: {0:.3f} (std: {1:.3f})".format(
              results['mean_test_score'][candidate],
              results['std_test_score'][candidate]))
        print("Parameters: {0}".format(results['params'][candidate]))
        print("")

print(report(gs.cv_results_,10))

epochs = [i for i in range(len(gs.best_estimator_.history))]
train_loss = gs.best_estimator_.history[:,'train_loss']
valid_loss = gs.best_estimator_.history[:,'valid_loss']

plt.plot(epochs,train_loss,'g-');
plt.plot(epochs,valid_loss,'r-');
plt.title('Training Loss Curves');
plt.xlabel('Epochs');
plt.ylabel('Mean Squared Error');
plt.legend(['Train','Validation']);
plt.show()

r = permutation_importance(net, X_test, y_test, n_repeats=30,random_state=0)

for i in r.importances_mean.argsort()[::-1]:
    if r.importances_mean[i] - 2 * r.importances_std[i] > 0:
        print(f"{metabolites.feature_names[i]:<8}"
              f"{r.importances_mean[i]:.3f}"
              f" +/- {r.importances_std[i]:.3f}")

y_pred_acc = gs.predict(X_test)
print('Accuracy : ' + str(accuracy_score(y_test,y_pred_acc)))

Stacktrace会指出错误源于我设置排列重要性的行。我怎样才能解决这个问题

完整堆栈跟踪：

*Traceback (most recent call last):
  File "//ad..fi/home/h//Desktop/neuralnet/neuralnet_wrapped.py", line 141, in <module>
    run()
  File "//ad..fi/home/h//Desktop/neuralnet/neuralnet_wrapped.py", line 119, in run
    r = permutation_importance(net, X_test, y_test,
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\utils\validation.py", line 73, in inner_f
    return f(**kwargs)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\inspection\_permutation_importance.py", line 132, in permutation_importance
    scorer = check_scoring(estimator, scoring=scoring)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\utils\validation.py", line 73, in inner_f
    return f(**kwargs)
  File "C:\Users\\AppData\Roaming\Python\Python38\site-packages\sklearn\metrics\_scorer.py", line 425, in check_scoring
    raise TypeError(
TypeError: If no scoring is specified, the estimator passed should have a 'score' method. The estimator <class 'skorch.net.NeuralNet'>[uninitialized](
  module=<class '__main__.run.<locals>.MultiLayerPredictor'>,
) does not.*

Answer 1

正如Berriel所说，这失败了，因为您的神经网络实例没有实现score()方法。这是默认值，因为不清楚对于任意学习任务应该返回什么分数

在sklearn网格搜索中也会发生这种情况，您通过传递scoring='neg_mean_squared_error'来避免这种情况。您也可以在此处执行此操作：

r = permutation_importance(net, X_test, y_test, 
        scoring='neg_mean_squared_error', n_repeats=30, random_state=0)

或者，假设您也需要在其他地方评分，您可以自己实现score方法：

class MyNet(NeuralNetwork):
    def score(self, X, y):
        y = self.predict(X)
        return sklearn.metrics.mean_squared_error(y, y_pred)