重塑数据帧并对每个帧应用计算

Question

我有一个数据框如下：

df=pd.DataFrame({ 'family' : ["A","A","B","B"],
  'V1' : [5,5,40,10,],
  'V2' :[50,10,180,20],
  'gr_0' :["all","all","all","all"],
  'gr_1' :["m1","m1","m2","m3"],
  'gr_2' :["m12","m12","m12","m9"],
  'gr_3' :["NO","m14","m15","NO"]
                           })

我想用以下方式来改变它：

df_new=pd.DataFrame({ 'family' : ["A","A","A","A","B","B","B","B","B","B"],
  'gr' : ["all","m1","m12","m14","all","m2","m3","m12","m9","m15"],
  "calc(sumV2/sumV1)":[6,6,6,2,4,4.5,2,4.5,2,4.5]            
               })

  family   gr  calc(sumV2/sumV1)
0      A  all                6.0
1      A   m1                6.0
2      A  m12                6.0
3      A  m14                2.0
4      B  all                4.0
5      B   m2                4.5
6      B   m3                2.0
7      B  m12                4.5
8      B   m9                2.0
9      B  m15                4.5

为了达到新的目的：

1. 我希望这些行按“family”X“gr\ux”列的每个唯一值对齐。你知道吗
2. 为每一行计算相应的和（V2）/和（V1），如df\u new中所示。你知道吗

我对Python很陌生。软编码对我来说似乎相当复杂。 最好，我不希望“No”记录被列在这个df\u new中，但它也可以保留在输出中。你知道吗

Answer 1

melt+groupby：

v = df.melt(id_vars=['family','V1','V2'], value_name='gr')
w = v.loc[v.gr != 'NO']
x = w.groupby(['family', 'gr']).sum()

(x.V2 / x.V1).reset_index(name='calc(sumV2/sumV1)')

  family   gr  calc(sumV2/sumV1)
0      A  all                6.0
1      A   m1                6.0
2      A  m12                6.0
3      A  m14                2.0
4      B  all                4.0
5      B  m12                4.5
6      B  m15                4.5
7      B   m2                4.5
8      B   m3                2.0
9      B   m9                2.0

---

一种类似于this answer的方法，但其优点是完全矢量化，避免了apply

---

性能：

a = np.random.randint(1, 1000, (1_000_000, 7))
df = pd.DataFrame(a, columns=['family', 'V1', 'V2', 'gr_0', 'gr_1', 'gr_2', 'gr_3'])   
df[['gr_0', 'gr_1', 'gr_2', 'gr_3']] = df[['gr_0', 'gr_1', 'gr_2', 'gr_3']].astype(str)

%%timeit
v = df.melt(id_vars=['family','V1','V2'], value_name='gr')
w = v.loc[v.gr != 'NO']
x = w.groupby(['family', 'gr']).sum()    
(x.V2 / x.V1).reset_index(name='calc(sumV2/sumV1)')

2.71 s ± 32.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%%timeit
df_new = (df.melt(id_vars=['family','V1','V2']).groupby(['family','value'])
                 .apply(lambda x: x.V2.sum()/x.V1.sum())
                 .reset_index(name='calc(sumV2/sumV1)'))
df_new = df_new[df_new.value != 'NO'].reset_index(drop=True)

5min 24s ± 3.35 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

Answer 2

您可以这样做：

df_new = df.melt(id_vars=['family','V1','V2']).groupby(['family','value'])
                .apply(lambda x: x.V2.sum()/x.V1.sum())
                .reset_index(name='calc(sumV2/sumV1)')
df_new = df_new[df_new.value != 'NO'].reset_index(drop=True)

print(df_new)

     family value  calc(sumV2/sumV1)
0      A    all           6.0
1      A    m1            6.0
2      A    m12           6.0
3      A    m14           2.0
4      B    all           4.0
5      B    m12           4.5
6      B    m15           4.5
7      B    m2            4.5
8      B    m3            2.0
9      B    m9            2.0