删除在1列中总和为零但在其他方面在1列中重复的行

Question

我有一个如下结构的数据框架：

df = pd.DataFrame({'ID':['A001', 'A001', 'A001', 'A002', 'A002', 'A003', 'A003', 'A004', 'A004', 'A004', 'A005', 'A005'],
                   'Val1':[2, 2, 2, 5, 6, 8, 8, 3, 3, 3, 7, 7],
                   'Val2':[100, -100, 50, -40, 40, 60, -50, 10, -10, 10, 15, 15]})

    ID    Val1  Val2
 0  A001     2   100
 1  A001     2  -100
 2  A001     2    50
 3  A002     5   -40
 4  A002     6    40
 5  A003     8    60
 6  A003     8   -50
 7  A004     3    10
 8  A004     3   -10
 9  A004     3    10
10  A005     7    15
11  A005     7    15

我想删除重复的行，其中ID和Val1是重复的，Val2在两行中的总和为零。正/负Val2行也可能不是连续的，即使在groupby下也是如此

在上面的示例数据中，行0和1以及7、8、9满足这些标准。我想删除[0,1]，并删除[7,8]或[8,9]

这里的另一个限制是可能存在完全重复的行（[10,11]）。在本例中，我希望保留两行

因此，期望输出为：

    ID    Val1  Val2
 2  A001     2    50
 3  A002     5   -40
 4  A002     6    40
 5  A003     8    60
 6  A003     8   -50
 9  A004     3    10
10  A005     7    15
11  A005     7    15

除了迭代每一行并寻找符合条件的其他行之外，我没有更“pythonic”的方法来实现这一点。非常感谢您的帮助

Answer 1

我在代码中添加了一些注释，因此希望我的思路应该是明确的：

cond = df.assign(temp=df.Val2.abs())
# a way to get the same values (differentiated by their sign)
# to follow each other
cond = cond.sort_values(["ID", "Val1", "temp"])

# cumsum should yield a zero for numbers that are different
# only by their sign
cond["check"] = cond.groupby(["ID", "temp"]).Val2.cumsum()
cond["check"] = np.where(cond.check != 0, np.nan, cond.check)

# the backward fill here allows us to assign an identifier
# to the two values that summed to zero
cond["check"] = cond["check"].bfill(limit=1)

# this is where we implement your other condition
# essentially, it looks for rows that are duplicates
# and rows that any two rows sum to zero
cond.loc[
    ~(cond.duplicated(["ID", "Val1"], keep=False) & (cond.check == 0)),
    ["ID", "Val1", "Val2"],
]



     ID Val1    Val2
2   A001    2   50
3   A002    5   -40
4   A002    6   40
6   A003    8   -50
5   A003    8   60
9   A004    3   10

Answer 2

那么：

temp = df.groupby('ID')[['Val2']].rolling(2).sum()
ix = temp[temp.Val2==0].index
ar = np.array([x[1] for x in ix.values])
ix2 = ar.tolist() + (ar-1).tolist()
df.drop(ix2, inplace=True)
df.drop_duplicates(['ID', 'Val1'], keep='first', inplace=True)

但这个答案指的是你的“文本”答案：第8行&；9‘Val2’实际上等于零（这不是您发布的“期望输出”

Answer 3

使用groupby和cumsum查找Val2的哪个索引和为零

s = df.groupby(['ID', 'Val1']).Val2.cumsum() == 0

n = np.where(s==1)[0]

to_remove = np.concatenate((n, (n-1))) 

new_df = df[~df.index.isin(to_remove)]

new_df 

      ID  Val1  Val2
2   A001     2    50
3   A002     5   -40
4   A002     6    40
5   A003     8    60
6   A003     8   -50
9   A004     3    10
10  A005     7    15
11  A005     7    15