在datafram中的集合中删除'NaN'值

Question

我有一个数据帧，看起来像这样，但有多行：

    column_1         column_2         column_3
        1         {lk, 18m, NaN}    {kjaf, NaN} 

我想从每个集合中取出NaN，但是遍历这些行会得到RuntimeError: Set changed size during iteration。你知道吗

我目前使用的代码如下：

for index, row in df.iterrows():
    col2 = row['column_2']
    col3 = row['column_3']
    for x in col2:
        col2.discard('NaN')
    for y in col3:
        col3.discard('NaN')

Answer 1

如果NaN缺少值，则可以在集合理解中使用if：

df = pd.DataFrame({'column_1': [1, 1], 
                   'column_2': [[np.nan, '18m'], ['lk', 'r']],
                   'column_3': [['kjaf'], ['ddd']]})

print (df)
   column_1    column_2 column_3
0         1  [nan, 18m]   [kjaf]
1         1     [lk, r]    [ddd]

cols = ['column_2', 'column_3']
df[cols] = df[cols].applymap(lambda x: set([i for i in x if pd.notna(i)]))
#oldier pandas versions
#df[cols] = df[cols].applymap(lambda x: set([i for i in x if pd.notnull(i)]))
print (df)

   column_1 column_2 column_3
0         1    {18m}   {kjaf}
1         1  {r, lk}    {ddd}

如果NaN是字符串：

df = pd.DataFrame({'column_1': [1, 1], 
                   'column_2': [['NaN', '18m'], ['lk', 'r']],
                   'column_3': [['kjaf'], ['ddd']]})

print (df)
   column_1    column_2 column_3
0         1  [NaN, 18m]   [kjaf]
1         1     [lk, r]    [ddd]

cols = ['column_2', 'column_3']
df[cols] = df[cols].applymap(lambda x: set([i for i in x if i != 'NaN']))
print (df)

   column_1 column_2 column_3
0         1    {18m}   {kjaf}
1         1  {r, lk}    {ddd}