如何在sklearn中使用make_scorer自定义评分功能

Question

我正在尝试实现一个最高十分位数召回/精确评分函数，以插入到gridsearchCV中。但是，我不知道出了什么问题。我想做的是让我的评分函数，在概率预测，实际标签和理想的十分位阈值百分比。然后我会对分数进行排序，然后在十分位数阈值内确定转换率。E、 g.前10%人口的转化率。转化率就是我输出的分数。越高越好。但是我不明白下面的函数是什么，我不知道什么是分数。下面的print语句只返回1和0，而不是概率。在

def top_decile_conversion_rate(y_prob, y_actual):
    # Function goes in here
    print y_prob, y_actual
    return 0.5


features = pd.DataFrame({"f1":np.random.randint(1,1000,500) , "f2":np.random.randint(1,1000,500), 
                         "label":[round(x) for x in np.random.random_sample(500)]})


my_scorer = make_scorer(top_decile_conversion_rate, greater_is_better=True)
gs = grid_search.GridSearchCV(
    estimator=LogisticRegression(),
    param_grid={'C': [i for i in range(1, 3)], 'class_weight': [None], 'penalty':['l2']},
    cv=2,
    scoring=my_scorer ) 
model = gs.fit(features[["f1","f2"]], features.label)

Answer 1

解决方法是在make\u scorer函数中添加一个名为needs_proba=True的参数！这行得通。在

def top_decile_conversion_rate(y_prob, y_actual):
    # Function goes in here
    print " -prob "
    print y_prob
    print " -actual "
    print y_actual
    print " -end "

    return 0.5


features = pd.DataFrame({"f1":np.random.randint(1,1000,500) , "f2":np.random.randint(1,1000,500), 
                         "label":[round(x) for x in np.random.random_sample(500)]})


my_scorer = make_scorer(top_decile_conversion_rate, greater_is_better=True,needs_proba=True)
gs = grid_search.GridSearchCV(
    estimator=LogisticRegression(),
    param_grid={'C': [i for i in range(1, 3)], 'class_weight': [None], 'penalty':['l2']},
    cv=20,
    scoring=my_scorer ) 
model = gs.fit(features[["f1","f2"]], features.label)