scikit学习转换多个文本功能

Question

我正在尝试将多个文本特征分类到一个状态。这些数据包括来自不同服务器和组件的消息（错误和警告），并将导致不同的状态。例如：

ServerName     Name     Description                               Severity   State
-------------- -------- ----------------------------------------- ---------- -------------
QWERT-XY-123   MySQL    Service not available on target machine   error      important
QWERT-XY-146   Oracle   Service caused an error                   warning    unimportant
...    

这是矢量化的一部分：

^{pr2}$

现在我要适合模型：

^{3}$

结果如下：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-183-71455dd49f0b> in <module>()
  2 
  3 model = SVC(kernel = "linear", probability=True)
----> 4 model.fit(features, y)
  5 
  6 #print(model.score(X_test, y))

D:\Enviroment\Anaconda3\lib\site-packages\sklearn\svm\base.py in fit(self, X, y, sample_weight)
147         self._sparse = sparse and not callable(self.kernel)
148 
149 -->     X, y = check_X_y(X, y, dtype=np.float64, order='C', accept_sparse='csr')
150         y = self._validate_targets(y)
151 

D:\Enviroment\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_X_y(X, y, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, warn_on_dtype, estimator)
571     X = check_array(X, accept_sparse, dtype, order, copy, force_all_finite,
572                     ensure_2d, allow_nd, ensure_min_samples,
573 -->                 ensure_min_features, warn_on_dtype, estimator)
574     if multi_output:
575         y = check_array(y, 'csr', force_all_finite=True, ensure_2d=False,

D:\Enviroment\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
431                                       force_all_finite)
432     else:
433 -->     array = np.array(array, dtype=dtype, order=order, copy=copy)
434 
435         if ensure_2d:

ValueError: setting an array element with a sequence.

所以我的问题是如何在hashingvectorizer中使用多个特性，还是将所有特性放在一行中的唯一方法？在

谢谢你的帮助。在

更新

失败的是如何建立矢量化特征列表。而不是：

features=list(zip(X_Servername,X_Name,X_Description,X_Severity))

我现在使用这个函数，其中extracted附加所有创建的矢量化值（X\u ServerName，X\u Name，…）：

def combine(extracted):
    if any(sparse.issparse(fea) for fea in extracted):
        stacked = sparse.hstack(extracted).tocsr()
        stacked = stacked.toarray()
    else:
        stacked = np.hstack(extracted)

    return stacked

Answer 1

请尝试以下代码：

from sklearn_pandas import DataFrameMapper, gen_features
from sklearn.feature_extraction.text import HashingVectorizer
from sklearn.preprocessing import LabelEncoder

cat_features = ["ServerName", "Name", "Description", "Severity"]
gf = gen_features(cat_features, [HashingVectorizer])
mapper = DataFrameMapper(gf)
cat_features_transformed = mapper.fit_transform(df)

target_name_encoded = LabelEncoder().fit_transform(df["State"])

from sklearn.svm import SVC

model = SVC(kernel = "linear", probability=True)
model.fit(cat_features_transformed, target_name_encoded)
SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',
  kernel='linear', max_iter=-1, probability=True, random_state=None,
  shrinking=True, tol=0.001, verbose=False)

### For test/prediction part ###

test_features_transformed = mapper.transform(df_test)
predictions = model.predict(test_features_transformed)

注意，您可能需要运行

^{pr2}$

如果您的计算机上没有安装sklearn-pandas。在

上述解决方案将允许您（1）将数据转换为合适的格式，然后（2）通过transform方法对测试数据应用相同的适合转换。在

请告诉我们这是否有帮助