XGBoost对list和array给出了稍微不同的预测，这是正确的吗？

Question

我注意到我传递了一个双括号的测试特性值列表

print(test_feats)
>> [[23.0, 3.0, 35.0, 0.28, -3.0, 18.0, 0.0, 0.0, 0.0, 3.33, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 39.0, 36.0, 113.0, 76.0, 0.0, 0.0, 1.0, 0.34, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, 0.0, 25.0, 48.0, 48.0, 0.0, 29.0, 52.0, 53.0, 99.0, 368.0, 676.0, 691.0, 4.0, 9.0, 12.0, 13.0]]

我注意到，当我将它传递给XBGBoost进行预测时，当我将它转换为数组时，它会返回不同的结果

array_test_feats = np.array(test_feats)
print(regr.predict_proba(test_feats)[:,1][0])
print(regr.predict_proba(aray_test_feats)[:,1][0])
>> 0.46929297
>> 0.5161868

一些基本检查表明值是相同的

print(sum(test_feats[0]) == array_test_feats.sum())
print(test_feats == array_test_feats)) 
>> True
>> array([[ True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True]])

我猜阵列是一种方法，但我真的不知道怎么说。这些预测非常接近，很容易被忽略，所以我真的很想理解为什么会发生这种情况

Answer 1

您刚刚遇到了这里描述的问题：https://github.com/dmlc/xgboost/pull/3970

The documentation does not include lists as an allowed type for the data inputted into DMatrix. Despite this, a list can be passed in without an error. This change would prevent a list form being passed in directly.

I experienced an issue where passing in a list vs a np.array resulted in different predictions (sometimes over 10% relative difference) for the same data. Though these differences were infrequent (~1.5% of cases tested), in certain applications this could cause serious issues.

从本质上讲，直接传递Python列表在XGBoost中是不受官方支持的，但是无论如何它都能工作，因为它在XGBoost的数据转换中命中了a fall through case

这导致XGBoost使用XGDMatrixCreateFromCSREx函数而不是XGDMatrixCreateFromMat来为数据创建underyling矩阵。然后在sprase和dense表示中缺少的元素之间有一个difference in behavior：

"Sparse" elements are treated as "missing" by the tree booster and as zeros by the linear booster.